diff mbox series

[ovs-dev,v1] Add support for centralize routing for distributed gw ports.

Message ID 20240709232515.2368930-1-numans@ovn.org
State Superseded
Headers show
Series [ovs-dev,v1] Add support for centralize routing for distributed gw ports. | expand

Checks

Context Check Description
ovsrobot/apply-robot success apply and check: success
ovsrobot/github-robot-_Build_and_Test success github build: passed
ovsrobot/github-robot-_ovn-kubernetes success github build: passed

Commit Message

Numan Siddique July 9, 2024, 11:25 p.m. UTC
From: Numan Siddique <numans@ovn.org>

Consider a deployment with the below logical resources:

1. A bridged logical switch 'public' with a port - P1 and a localnet
   port ln-public.
2. A logical router 'R'
3. Logical switch 'public' connected to R via logical switch/router port
   peers (public-R and R-public).
4. R-public is distributed gateway port with its network as 172.16.0.0/24
5. NATs (dnat_and_snat) configured in 'R'.
6. And a few overlay logical switches S1, S2 to R.

Any traffic from logical port - P1 of public logical switch destined to
S1 or S2's logical ports goes out of the source chassis
(where P1 resides) via the localnet port and reaches the gateway chassis
which handles the routing.

There are couple of traffic flow scenarios which doesn't work if the
logical switch 'public' doesn't have a localnet port.

1. Traffic from port - P1 destined to logical switches S1 or S2 gets
   dropped in the source chassis.  The packet enters the router R's
   pipeline, but it gets dropped in the 'lr_in_admission' stage since
   the logical flow to allow traffic destined to the distributed gateway
   port MAC is installed only on the gateway chassis.

2. NAT doesn't work as expected.

In order to suppose this use case (of a logical switch not having a
localnet port, but has a distributed gateway port and NATs), this patch
supports the option 'centralize_routing', which can be configured on
the distributed gateway port (R-public in the example above).
If this option is set, then routing is centralized on the gateway
chassis for the traffic destined to the R-public's networks
(172.16.0.0/24 for the above example).  Traffic from P1 will be
tunnelled to the gateway chassis.

ovn-northd creates a chassisresident port (cr-public-R) for the
logical switch port - public-R, along with cr-R-public inorder to
centralize the traffic.

This feature gets enabled for the distributed gateway port R-public if
  - The above option is set to true in the R-public's options column.
  - The logical switch 'public' doesn't have any localnet ports.
  - And R-public is the only distributed gateway port of R.

Distributed NAT (i.e if external_mac and router_port is set) is
not supported and instead the router port mac is used for such traffic
and centralized on the gateway chassis.

Reported-at: https://issues.redhat.com/browse/FDP-364
Signed-off-by: Numan Siddique <numans@ovn.org>
---
 NEWS                      |   2 +
 controller/physical.c     |   4 +
 northd/northd.c           | 257 +++++++++++++++----
 northd/northd.h           |   1 +
 ovn-nb.xml                |  34 +++
 tests/multinode-macros.at |   2 +-
 tests/multinode.at        | 177 +++++++++++++
 tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
 tests/ovn.at              |   8 +-
 9 files changed, 951 insertions(+), 50 deletions(-)

Comments

Numan Siddique July 25, 2024, 2:12 p.m. UTC | #1
On Tue, Jul 9, 2024 at 7:25 PM <numans@ovn.org> wrote:
>
> From: Numan Siddique <numans@ovn.org>
>
> Consider a deployment with the below logical resources:
>
> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>    port ln-public.
> 2. A logical router 'R'
> 3. Logical switch 'public' connected to R via logical switch/router port
>    peers (public-R and R-public).
> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> 5. NATs (dnat_and_snat) configured in 'R'.
> 6. And a few overlay logical switches S1, S2 to R.
>
> Any traffic from logical port - P1 of public logical switch destined to
> S1 or S2's logical ports goes out of the source chassis
> (where P1 resides) via the localnet port and reaches the gateway chassis
> which handles the routing.
>
> There are couple of traffic flow scenarios which doesn't work if the
> logical switch 'public' doesn't have a localnet port.
>
> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>    dropped in the source chassis.  The packet enters the router R's
>    pipeline, but it gets dropped in the 'lr_in_admission' stage since
>    the logical flow to allow traffic destined to the distributed gateway
>    port MAC is installed only on the gateway chassis.
>
> 2. NAT doesn't work as expected.
>
> In order to suppose this use case (of a logical switch not having a
> localnet port, but has a distributed gateway port and NATs), this patch
> supports the option 'centralize_routing', which can be configured on
> the distributed gateway port (R-public in the example above).
> If this option is set, then routing is centralized on the gateway
> chassis for the traffic destined to the R-public's networks
> (172.16.0.0/24 for the above example).  Traffic from P1 will be
> tunnelled to the gateway chassis.
>
> ovn-northd creates a chassisresident port (cr-public-R) for the
> logical switch port - public-R, along with cr-R-public inorder to
> centralize the traffic.
>
> This feature gets enabled for the distributed gateway port R-public if
>   - The above option is set to true in the R-public's options column.
>   - The logical switch 'public' doesn't have any localnet ports.
>   - And R-public is the only distributed gateway port of R.
>
> Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic
> and centralized on the gateway chassis.
>
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>  NEWS                      |   2 +
>  controller/physical.c     |   4 +
>  northd/northd.c           | 257 +++++++++++++++----
>  northd/northd.h           |   1 +
>  ovn-nb.xml                |  34 +++
>  tests/multinode-macros.at |   2 +-
>  tests/multinode.at        | 177 +++++++++++++
>  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>  tests/ovn.at              |   8 +-
>  9 files changed, 951 insertions(+), 50 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 3e392ff08b..472445a188 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -38,6 +38,8 @@ Post v24.03.0
>      ability to disable "VXLAN mode" to extend available tunnel IDs space for
>      datapaths from 4095 to 16711680.  For more details see man ovn-nb(5) for
>      mentioned option.
> +  - Added Overlay provider network support to a logical switch if
> +    the config "overlay_provider_network" is set to true.

Please ignore this modification to NEWS.  It's wrong.  I'll fix it in
the next version.

Numan

>
>  OVN v24.03.0 - 01 Mar 2024
>  --------------------------
> diff --git a/controller/physical.c b/controller/physical.c
> index 22756810fd..e3a316989a 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1608,6 +1608,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                      ct_zones);
>              put_zones_ofpacts(&zone_ids, ofpacts_p);
>
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>              /* Resubmit to table 41. */
>              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>          }
> diff --git a/northd/northd.c b/northd/northd.c
> index 6898daa00d..9b52d5a3c0 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2099,6 +2099,55 @@ parse_lsp_addrs(struct ovn_port *op)
>      }
>  }
>
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports,
> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> +        ovn_port_set_nb(crp, NULL, op->nbrp);
> +        ovs_list_remove(&crp->list);
> +        ovs_list_push_back(both_dbs, &crp->list);
> +    } else {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +        ovs_list_push_back(nb_only, &crp->list);
> +    }
> +
> +    crp->primary_port = op;
> +    op->cr_port = crp;
> +    crp->od = op->od;
> +    free(redirect_name);
> +
> +    return crp;
> +}
> +
> +/* Returns true if chassis resident port needs to be created for
> + * op's peer logical switch.  False otherwise.
> + *
> + * Chassis resident port needs to be created if the following
> + * conditionsd are met:
> + *   - op is a distributed gateway port
> + *   - op has the option 'centralize_routing' set to true
> + *   - op is the only distributed gateway port attached to its
> + *     router
> + *   - op's peer logical switch has no localnet ports.
> + */
> +static bool
> +peer_needs_cr_port_creation(struct ovn_port *op)
> +{
> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +        && !op->peer->od->n_localnet_ports) {
> +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> +    }
> +
> +    return false;
> +}
> +
>  static void
>  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2206,9 +2255,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>          }
>      }
> +
> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>          ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>          for (size_t i = 0; i < od->nbr->n_ports; i++) {
>              const struct nbrec_logical_router_port *nbrp
>                  = od->nbr->ports[i];
> @@ -2272,10 +2322,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      redirect_type && !strcasecmp(redirect_type, "bridged");
>              }
>
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                  const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                 "chassis");
>                  if (gw_chassis) {
> @@ -2284,34 +2331,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                   "gateway port configured on port %s "
>                                   "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                  } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->primary_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&dgps, op);
>                  }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>             }
>          }
>      }
> @@ -2368,12 +2390,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                          arp_proxy, op->nbsp->name);
>                  }
>              }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>              if (peer) {
> @@ -2394,6 +2410,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>          }
>      }
>
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> +        ovs_assert(crp);
> +
> +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                        &od->n_allocated_l3dgw_ports,
> +                                        sizeof *od->l3dgw_ports);
> +        }
> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the distributed gateway port's (DGP)
> +     * peer if
> +     *  - DGP's router has only one DGP and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports and
> +     *  - option 'centralize_routing' is set to true for the DGP.
> +     *
> +     * This is required to support
> +     *   - NAT via geneve (for the overlay provider networks) and
> +     *   - to centralize routing on the gateway chassis for the traffic
> +     *     destined to the DGP's networks.
> +     *
> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> +     * of a logical router.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        if (peer_needs_cr_port_creation(op)) {
> +            create_cr_port(op->peer, ports, both, nb_only);
> +        }
> +    }
> +    hmapx_destroy(&dgps);
> +
>      /* Wait until all ports have been connected to add to IPAM since
>       * it relies on proper peers to be set
>       */
> @@ -3176,16 +3243,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>               * type "l3gateway". */
>              if (chassis) {
>                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->primary_port->peer);
> +                ovs_assert(op->primary_port->peer->cr_port);
> +                ovs_assert(op->primary_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> +
>              } else {
>                  sbrec_port_binding_set_type(op->sb, "patch");
>              }
>
>              const char *router_port = smap_get(&op->nbsp->options,
>                                                 "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                  struct smap new;
>                  smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                      smap_add(&new, "peer", router_port);
>                  }
>                  if (chassis) {
> @@ -8191,9 +8270,27 @@ build_lswitch_rport_arp_req_flow(
>      struct lflow_ref *lflow_ref)
>  {
>      struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>      struct ds actions = DS_EMPTY_INITIALIZER;
>
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    /* If the patch_op has a chassis resident port, it means
> +     *    - its peer is a distributed gateway port (DGP) and
> +     *    - routing is centralized for the DGP's networks on
> +     *      the configured gateway chassis.
> +     *
> +     * If that's the case, make sure that the packets destined to
> +     * the DGP's MAC are sent to the chassis where the DGP resides.
> +     * */
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>
>      /* Send a the packet to the router pipeline.  If the switch has non-router
>       * ports then flood it there as well.
> @@ -8215,6 +8312,31 @@ build_lswitch_rport_arp_req_flow(
>                                  lflow_ref);
>      }
>
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>      ds_destroy(&match);
>      ds_destroy(&actions);
>  }
> @@ -9585,7 +9707,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                  struct ds *actions, struct ds *match)
>  {
>      ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +
> +    /* Note: A switch port can also have a chassis resident derived port.
> +     * Check if 'op' is a chassis resident dervied port. If so, skip
> +     * adding unicast lookup flows for this port. */
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>          return;
>      }
>
> @@ -9603,8 +9729,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                             "outport = \""MC_UNKNOWN "\"; output;"
>                           : "outport = %s; output;")
>                           : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>
>      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>          /* For ports connected to logical routers add flows to bypass the
> @@ -9651,14 +9775,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>              if (add_chassis_resident_check) {
>                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>              }
> +        } else if (op->cr_port) {
> +            /* If the op has a chassis resident port, it means
> +             *   - its peer is a distributed gateway port (DGP) and
> +             *   - routing is centralized for the DGP's networks on
> +             *     the configured gateway chassis.
> +             *
> +             * If that's the case, make sure that the packets destined to
> +             * the DGP's MAC are sent to the chassis where the DGP resides.
> +             * */
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>          }
>
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          ovn_lflow_add_with_hint(lflows, op->od,
>                                  S_SWITCH_IN_L2_LKUP, 50,
>                                  ds_cstr(match), ds_cstr(actions),
>                                  &op->nbsp->header_,
>                                  op->lflow_ref);
>      } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>              ds_clear(match);
>              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11772,6 +11925,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>          return;
>      }
>
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the router port networks/subnets.
> +         */
> +        return;
> +    }
> +
>      /* Mac address to use when replying to ARP/NS. */
>      const char *mac_s = REG_INPORT_ETH_ADDR;
>      struct eth_addr mac;
> @@ -15158,6 +15319,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>      /* For distributed router NAT, determine whether this NAT rule
>       * satisfies the conditions for distributed NAT processing. */
>      *distributed = false;
> +
> +    /* NAT cannnot be distributed if the DGP's peer
> +     * has a chassisresident port (as the routing is centralized
> +     * on the gateway chassis for the DGP's networks/subnets.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>          nat->logical_port && nat->external_mac) {
>          if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index d4a8d75abc..d7c9655916 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>       * will be NULL. */
>      struct ovn_port **l3dgw_ports;
>      size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>
>      /* router datapath has a logical port with redirect-type set to bridged. */
>      bool redirect_bridged;
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index 9552534f6d..794e2fb961 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -3451,6 +3451,40 @@ or
>            <ref column="options" key="gateway_mtu"/> option.
>          </p>
>        </column>
> +
> +      <column name="options" key="centralize_routing"
> +              type='{"type": "boolean"}'>
> +        <p>
> +          This option is applicable only if the router port is a
> +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> +          column="ha_chassis_group"/> column or
> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> +          is set.
> +        </p>
> +
> +        <p>
> +          If set to <code>true</code>, routing for the router port's
> +          networks (set in the column <ref table="Logical_Router_Port"
> +          column="networks"/>) is centralized on the gateway chassis
> +          which claims this distributed gateway port.
> +        </p>
> +
> +        <p>
> +          Additionally for this option to take effect, below conditions
> +          must be met:
> +        </p>
> +
> +        <ul>
> +          <li>
> +            The Logical router has only one distributed gateway port.
> +          </li>
> +
> +          <li>
> +            The router port's peer logical switch has no localnet ports.
> +          </li>
> +
> +        </ul>
> +      </column>
>      </group>
>
>      <group title="Attachment">
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index ef41087ae3..df99f01b64 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -73,7 +73,7 @@ m_count_rows() {
>  m_check_row_count() {
>      local db=$(parse_db $1) table=$(parse_table $1); shift
>      local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>      echo
>      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>      if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index 1e6eeb6610..9e01a29cc2 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1033,4 +1033,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
>  done
>  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
>
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# centralize_routing=true is not yet set for lr0-public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option - centralize_routing now.
> +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>  AT_CLEANUP
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index a389d19886..5445fff494 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2188,7 +2188,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>  ])
>
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>
>  AT_CLEANUP
> @@ -5524,13 +5524,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>
>  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>
>  ovn-sbctl lflow-list ls1 > ls1_lflows
>  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12721,3 +12722,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
>
>  AT_CLEANUP
>  ])
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([NAT])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "centralize_routing=true" for lr0-public.
> +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 185ba4a21e..21d7484f9e 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21232,10 +21232,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>      type=router options:router-port=sw0 \
>      -- lsp-set-addresses rp-sw0 router
>
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>      -- lsp-set-addresses rp-sw1 router
>
>  ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21247,6 +21247,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>  ovn-nbctl lsp-add sw1 sw1-p0 \
>      -- lsp-set-addresses sw1-p0 unknown
>
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>
> --
> 2.45.2
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Roberto Bartzen Acosta July 25, 2024, 3:20 p.m. UTC | #2
Em qui., 25 de jul. de 2024 às 11:12, Numan Siddique <numans@ovn.org>
escreveu:

> On Tue, Jul 9, 2024 at 7:25 PM <numans@ovn.org> wrote:
> >
> > From: Numan Siddique <numans@ovn.org>
> >
> > Consider a deployment with the below logical resources:
> >
> > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> >    port ln-public.
> > 2. A logical router 'R'
> > 3. Logical switch 'public' connected to R via logical switch/router port
> >    peers (public-R and R-public).
> > 4. R-public is distributed gateway port with its network as
> 172.16.0.0/24
> > 5. NATs (dnat_and_snat) configured in 'R'.
> > 6. And a few overlay logical switches S1, S2 to R.
> >
> > Any traffic from logical port - P1 of public logical switch destined to
> > S1 or S2's logical ports goes out of the source chassis
> > (where P1 resides) via the localnet port and reaches the gateway chassis
> > which handles the routing.
> >
> > There are couple of traffic flow scenarios which doesn't work if the
> > logical switch 'public' doesn't have a localnet port.
> >
> > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> >    dropped in the source chassis.  The packet enters the router R's
> >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> >    the logical flow to allow traffic destined to the distributed gateway
> >    port MAC is installed only on the gateway chassis.
> >
> > 2. NAT doesn't work as expected.
> >
> > In order to suppose this use case (of a logical switch not having a
> > localnet port, but has a distributed gateway port and NATs), this patch
> > supports the option 'centralize_routing', which can be configured on
> > the distributed gateway port (R-public in the example above).
> > If this option is set, then routing is centralized on the gateway
> > chassis for the traffic destined to the R-public's networks
> > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > tunnelled to the gateway chassis.
> >
> > ovn-northd creates a chassisresident port (cr-public-R) for the
> > logical switch port - public-R, along with cr-R-public inorder to
> > centralize the traffic.
> >
> > This feature gets enabled for the distributed gateway port R-public if
> >   - The above option is set to true in the R-public's options column.
> >   - The logical switch 'public' doesn't have any localnet ports.
> >   - And R-public is the only distributed gateway port of R.
> >
> > Distributed NAT (i.e if external_mac and router_port is set) is
> > not supported and instead the router port mac is used for such traffic
> > and centralized on the gateway chassis.
> >
> > Reported-at: https://issues.redhat.com/browse/FDP-364
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >  NEWS                      |   2 +
> >  controller/physical.c     |   4 +
> >  northd/northd.c           | 257 +++++++++++++++----
> >  northd/northd.h           |   1 +
> >  ovn-nb.xml                |  34 +++
> >  tests/multinode-macros.at |   2 +-
> >  tests/multinode.at        | 177 +++++++++++++
> >  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
> >  tests/ovn.at              |   8 +-
> >  9 files changed, 951 insertions(+), 50 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index 3e392ff08b..472445a188 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -38,6 +38,8 @@ Post v24.03.0
> >      ability to disable "VXLAN mode" to extend available tunnel IDs
> space for
> >      datapaths from 4095 to 16711680.  For more details see man
> ovn-nb(5) for
> >      mentioned option.
> > +  - Added Overlay provider network support to a logical switch if
> > +    the config "overlay_provider_network" is set to true.
>
> Please ignore this modification to NEWS.  It's wrong.  I'll fix it in
> the next version.
>
> Numan
>
> >
> >  OVN v24.03.0 - 01 Mar 2024
> >  --------------------------
> > diff --git a/controller/physical.c b/controller/physical.c
> > index 22756810fd..e3a316989a 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -1608,6 +1608,10 @@ consider_port_binding(struct ovsdb_idl_index
> *sbrec_port_binding_by_name,
> >                                                      ct_zones);
> >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> >
> > +            /* Clear the MFF_INPORT.  Its possible that the same packet
> may
> > +             * go out from the same tunnel inport. */
> > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16,
> ofpacts_p);
> > +
> >              /* Resubmit to table 41. */
> >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >          }
> > diff --git a/northd/northd.c b/northd/northd.c
> > index 6898daa00d..9b52d5a3c0 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -2099,6 +2099,55 @@ parse_lsp_addrs(struct ovn_port *op)
> >      }
> >  }
> >
> > +static struct ovn_port *
> > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > +{
> > +    char *redirect_name = ovn_chassis_redirect_name(
> > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > +
> > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > +        ovn_port_set_nb(crp, NULL, op->nbrp);
> > +        ovs_list_remove(&crp->list);
> > +        ovs_list_push_back(both_dbs, &crp->list);
> > +    } else {
> > +        crp = ovn_port_create(ports, redirect_name,
> > +                              op->nbsp, op->nbrp, NULL);
> > +        ovs_list_push_back(nb_only, &crp->list);
> > +    }
> > +
> > +    crp->primary_port = op;
> > +    op->cr_port = crp;
> > +    crp->od = op->od;
> > +    free(redirect_name);
> > +
> > +    return crp;
> > +}
> > +
> > +/* Returns true if chassis resident port needs to be created for
> > + * op's peer logical switch.  False otherwise.
> > + *
> > + * Chassis resident port needs to be created if the following
> > + * conditionsd are met:
> > + *   - op is a distributed gateway port
> > + *   - op has the option 'centralize_routing' set to true
> > + *   - op is the only distributed gateway port attached to its
> > + *     router
> > + *   - op's peer logical switch has no localnet ports.
> > + */
> > +static bool
> > +peer_needs_cr_port_creation(struct ovn_port *op)
> > +{
> > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > +        && !op->peer->od->n_localnet_ports) {
> > +        return smap_get_bool(&op->nbrp->options, "centralize_routing",
> false);
> > +    }
> > +
> > +    return false;
> > +}
> > +
> >  static void
> >  join_logical_ports(const struct sbrec_port_binding_table
> *sbrec_pb_table,
> >                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > @@ -2206,9 +2255,10 @@ join_logical_ports(const struct
> sbrec_port_binding_table *sbrec_pb_table,
> >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> >          }
> >      }
> > +
> > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> >          ovs_assert(od->nbr);
> > -        size_t n_allocated_l3dgw_ports = 0;
> >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> >              const struct nbrec_logical_router_port *nbrp
> >                  = od->nbr->ports[i];
> > @@ -2272,10 +2322,7 @@ join_logical_ports(const struct
> sbrec_port_binding_table *sbrec_pb_table,
> >                      redirect_type && !strcasecmp(redirect_type,
> "bridged");
> >              }
> >
> > -            if (op->nbrp->ha_chassis_group ||
> > -                op->nbrp->n_gateway_chassis) {
> > -                /* Additional "derived" ovn_port crp represents the
> > -                 * instance of op on the gateway chassis. */
> > +            if (op->nbrp->ha_chassis_group ||
> op->nbrp->n_gateway_chassis) {
> >                  const char *gw_chassis = smap_get(&op->od->nbr->options,
> >                                                 "chassis");
> >                  if (gw_chassis) {
> > @@ -2284,34 +2331,9 @@ join_logical_ports(const struct
> sbrec_port_binding_table *sbrec_pb_table,
> >                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> >                                   "gateway port configured on port %s "
> >                                   "on L3 gateway router", nbrp->name);
> > -                    continue;
> > -                }
> > -
> > -                char *redirect_name =
> > -                    ovn_chassis_redirect_name(nbrp->name);
> > -                struct ovn_port *crp = ovn_port_find(ports,
> redirect_name);
> > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > -                    ovs_list_remove(&crp->list);
> > -                    ovs_list_push_back(both, &crp->list);
> >                  } else {
> > -                    crp = ovn_port_create(ports, redirect_name,
> > -                                          NULL, nbrp, NULL);
> > -                    ovs_list_push_back(nb_only, &crp->list);
> > -                }
> > -                crp->primary_port = op;
> > -                op->cr_port = crp;
> > -                crp->od = od;
> > -                free(redirect_name);
> > -
> > -                /* Add to l3dgw_ports in od, for later use during flow
> > -                 * creation. */
> > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > -
>  &n_allocated_l3dgw_ports,
> > -                                                 sizeof
> *od->l3dgw_ports);
> > +                    hmapx_add(&dgps, op);
> >                  }
> > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> >             }
> >          }
> >      }
> > @@ -2368,12 +2390,6 @@ join_logical_ports(const struct
> sbrec_port_binding_table *sbrec_pb_table,
> >                          arp_proxy, op->nbsp->name);
> >                  }
> >              }
> > -
> > -            /* Only used for the router type LSP whose peer is
> l3dgw_port */
> > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > -                op->enable_router_port_acl = smap_get_bool(
> > -                    &op->nbsp->options, "enable_router_port_acl",
> false);
> > -            }
> >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> >              struct ovn_port *peer = ovn_port_find(ports,
> op->nbrp->peer);
> >              if (peer) {
> > @@ -2394,6 +2410,57 @@ join_logical_ports(const struct
> sbrec_port_binding_table *sbrec_pb_table,
> >          }
> >      }
> >
> > +    struct hmapx_node *hmapx_node;
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        od = op->od;
> > +        ovs_assert(op->nbrp);
> > +        ovs_assert(op->nbrp->ha_chassis_group ||
> op->nbrp->n_gateway_chassis);
> > +
> > +        /* Additional "derived" ovn_port crp represents the instance of
> op on
> > +         * the gateway chassis. */
> > +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> > +        ovs_assert(crp);
> > +
> > +        /* Add to l3dgw_ports in od, for later use during flow
> creation. */
> > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > +                                        &od->n_allocated_l3dgw_ports,
> > +                                        sizeof *od->l3dgw_ports);
> > +        }
> > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > +
> > +        if (op->peer && op->peer->nbsp) {
> > +            /* Only used for the router type LSP whose peer is
> l3dgw_port */
> > +            op->peer->enable_router_port_acl = smap_get_bool(
> > +                    &op->peer->nbsp->options, "enable_router_port_acl",
> false);
> > +        }
> > +    }
> > +
> > +
> > +    /* Create chassisresident port for the distributed gateway port's
> (DGP)
> > +     * peer if
> > +     *  - DGP's router has only one DGP and
> > +     *  - Its peer is a logical switch port and
> > +     *  - It's peer's logical switch has no localnet ports and
> > +     *  - option 'centralize_routing' is set to true for the DGP.
> > +     *
> > +     * This is required to support
> > +     *   - NAT via geneve (for the overlay provider networks) and
> > +     *   - to centralize routing on the gateway chassis for the traffic
> > +     *     destined to the DGP's networks.
> > +     *
> > +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> > +     * of a logical router.
> > +     * */
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        if (peer_needs_cr_port_creation(op)) {
> > +            create_cr_port(op->peer, ports, both, nb_only);
> > +        }
> > +    }
> > +    hmapx_destroy(&dgps);
> > +
> >      /* Wait until all ports have been connected to add to IPAM since
> >       * it relies on proper peers to be set
> >       */
> > @@ -3176,16 +3243,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn
> *ovnsb_txn,
> >               * type "l3gateway". */
> >              if (chassis) {
> >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > +            } else if (is_cr_port(op)) {
> > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > +                ovs_assert(op->primary_port->peer);
> > +                ovs_assert(op->primary_port->peer->cr_port);
> > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > +                sbrec_port_binding_set_ha_chassis_group(
> > +                    op->sb,
> > +
> op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > +
> >              } else {
> >                  sbrec_port_binding_set_type(op->sb, "patch");
> >              }
> >
> >              const char *router_port = smap_get(&op->nbsp->options,
> >                                                 "router-port");
> > -            if (router_port || chassis) {
> > +            if (router_port || chassis || is_cr_port(op)) {
> >                  struct smap new;
> >                  smap_init(&new);
> > -                if (router_port) {
> > +
> > +                if (is_cr_port(op)) {
> > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > +                } else if (router_port) {
> >                      smap_add(&new, "peer", router_port);
> >                  }
> >                  if (chassis) {
> > @@ -8191,9 +8270,27 @@ build_lswitch_rport_arp_req_flow(
> >      struct lflow_ref *lflow_ref)
> >  {
> >      struct ds match   = DS_EMPTY_INITIALIZER;
> > +    struct ds m       = DS_EMPTY_INITIALIZER;
> >      struct ds actions = DS_EMPTY_INITIALIZER;
> >
> > -    arp_nd_ns_match(ips, addr_family, &match);
> > +    arp_nd_ns_match(ips, addr_family, &m);
> > +    ds_clone(&match, &m);
> > +
> > +    bool has_cr_port = patch_op->cr_port;
> > +
> > +    /* If the patch_op has a chassis resident port, it means
> > +     *    - its peer is a distributed gateway port (DGP) and
> > +     *    - routing is centralized for the DGP's networks on
> > +     *      the configured gateway chassis.
> > +     *
> > +     * If that's the case, make sure that the packets destined to
> > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > +     * */
> > +
> > +    if (has_cr_port) {
> > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                      patch_op->cr_port->json_key);
> > +    }
> >
> >      /* Send a the packet to the router pipeline.  If the switch has
> non-router
> >       * ports then flood it there as well.
> > @@ -8215,6 +8312,31 @@ build_lswitch_rport_arp_req_flow(
> >                                  lflow_ref);
> >      }
> >
> > +    if (has_cr_port) {
> > +        ds_clear(&match);
> > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)",
> ds_cstr(&m),
> > +                      patch_op->cr_port->json_key);
> > +        ds_clear(&actions);
> > +        if (od->n_router_ports != od->nbs->n_ports) {
> > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > +                                    "outport = \""MC_FLOOD_L2"\";
> output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions), stage_hint,
> > +                                    lflow_ref);
> > +        } else {
> > +            ds_put_format(&actions, "outport = %s; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions),
> > +                                    stage_hint,
> > +                                    lflow_ref);
> > +        }
> > +    }
> > +
> > +    ds_destroy(&m);
> >      ds_destroy(&match);
> >      ds_destroy(&actions);
> >  }
> > @@ -9585,7 +9707,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> *op,
> >                                  struct ds *actions, struct ds *match)
> >  {
> >      ovs_assert(op->nbsp);
> > -    if (lsp_is_external(op->nbsp)) {
> > +
> > +    /* Note: A switch port can also have a chassis resident derived
> port.
> > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > +     * adding unicast lookup flows for this port. */
> > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> >          return;
> >      }
> >
> > @@ -9603,8 +9729,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> *op,
> >                             "outport = \""MC_UNKNOWN "\"; output;"
> >                           : "outport = %s; output;")
> >                           : debug_drop_action();
> > -    ds_clear(actions);
> > -    ds_put_format(actions, action, op->json_key);
> >
> >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> >          /* For ports connected to logical routers add flows to bypass
> the
> > @@ -9651,14 +9775,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> *op,
> >              if (add_chassis_resident_check) {
> >                  ds_put_format(match, " && is_chassis_resident(%s)",
> json_key);
> >              }
> > +        } else if (op->cr_port) {
> > +            /* If the op has a chassis resident port, it means
> > +             *   - its peer is a distributed gateway port (DGP) and
> > +             *   - routing is centralized for the DGP's networks on
> > +             *     the configured gateway chassis.
> > +             *
> > +             * If that's the case, make sure that the packets destined
> to
> > +             * the DGP's MAC are sent to the chassis where the DGP
> resides.
> > +             * */
> > +            ds_clear(actions);
> > +            ds_put_format(actions, action, op->cr_port->json_key);
> > +
> > +            struct ds m = DS_EMPTY_INITIALIZER;
> > +            ds_put_format(&m, "eth.dst == %s &&
> !is_chassis_resident(%s)",
> > +                          op->peer->lrp_networks.ea_s,
> > +                          op->cr_port->json_key);
> > +
> > +            ovn_lflow_add_with_hint(lflows, op->od,
> > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > +                                    ds_cstr(&m), ds_cstr(actions),
> > +                                    &op->nbsp->header_,
> > +                                    op->lflow_ref);
> > +            ds_destroy(&m);
> > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > +                          op->cr_port->json_key);
> >          }
> >
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          ovn_lflow_add_with_hint(lflows, op->od,
> >                                  S_SWITCH_IN_L2_LKUP, 50,
> >                                  ds_cstr(match), ds_cstr(actions),
> >                                  &op->nbsp->header_,
> >                                  op->lflow_ref);
> >      } else {
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> >              ds_clear(match);
> >              ds_put_format(match, "eth.dst == %s",
> op->lsp_addrs[i].ea_s);
> > @@ -11772,6 +11925,14 @@ build_lrouter_port_nat_arp_nd_flow(struct
> ovn_port *op,
> >          return;
> >      }
> >
> > +    if (op->peer && op->peer->cr_port) {
> > +        /* We don't add the below flows if the router port's peer has
> > +         * a chassisresident port.  That's because routing is
> centralized on
> > +         * the gateway chassis for the router port networks/subnets.
> > +         */
> > +        return;
> > +    }
> > +
> >      /* Mac address to use when replying to ARP/NS. */
> >      const char *mac_s = REG_INPORT_ETH_ADDR;
> >      struct eth_addr mac;
> > @@ -15158,6 +15319,16 @@ lrouter_check_nat_entry(const struct
> ovn_datapath *od,
> >      /* For distributed router NAT, determine whether this NAT rule
> >       * satisfies the conditions for distributed NAT processing. */
> >      *distributed = false;
> > +
> > +    /* NAT cannnot be distributed if the DGP's peer
> > +     * has a chassisresident port (as the routing is centralized
> > +     * on the gateway chassis for the DGP's networks/subnets.)
> > +     */
>

Hi Numan,

I assume that this implementation as a whole will not affect the stateless
use case in which the logical_router has multiple DGPs for different
ovn-chassis (chassisresident port), and with a stateless NAT rule so that
the traffic is decentralized via ECMP, right?

I mean, if I have a dgw_port peer for each chassis and configure a
stateless NAT on the router, I will continue creating the static DNAT and
SNAT path regardless of the port/chassis that the traffic passes through,
right? otherwise, this will make it impossible to balance traffic across
different chassis for the same logical_router.

Kind regards,
Roberto


> > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > +        return 0;
> > +    }
> > +
> >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> >          nat->logical_port && nat->external_mac) {
> >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > diff --git a/northd/northd.h b/northd/northd.h
> > index d4a8d75abc..d7c9655916 100644
> > --- a/northd/northd.h
> > +++ b/northd/northd.h
> > @@ -325,6 +325,7 @@ struct ovn_datapath {
> >       * will be NULL. */
> >      struct ovn_port **l3dgw_ports;
> >      size_t n_l3dgw_ports;
> > +    size_t n_allocated_l3dgw_ports;
> >
> >      /* router datapath has a logical port with redirect-type set to
> bridged. */
> >      bool redirect_bridged;
> > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > index 9552534f6d..794e2fb961 100644
> > --- a/ovn-nb.xml
> > +++ b/ovn-nb.xml
> > @@ -3451,6 +3451,40 @@ or
> >            <ref column="options" key="gateway_mtu"/> option.
> >          </p>
> >        </column>
> > +
> > +      <column name="options" key="centralize_routing"
> > +              type='{"type": "boolean"}'>
> > +        <p>
> > +          This option is applicable only if the router port is a
> > +          distributed gateway port i.e if the <ref
> table="Logical_Router_Port"
> > +          column="ha_chassis_group"/> column or
> > +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> > +          is set.
> > +        </p>
> > +
> > +        <p>
> > +          If set to <code>true</code>, routing for the router port's
> > +          networks (set in the column <ref table="Logical_Router_Port"
> > +          column="networks"/>) is centralized on the gateway chassis
> > +          which claims this distributed gateway port.
> > +        </p>
> > +
> > +        <p>
> > +          Additionally for this option to take effect, below conditions
> > +          must be met:
> > +        </p>
> > +
> > +        <ul>
> > +          <li>
> > +            The Logical router has only one distributed gateway port.
> > +          </li>
> > +
> > +          <li>
> > +            The router port's peer logical switch has no localnet ports.
> > +          </li>
> > +
> > +        </ul>
> > +      </column>
> >      </group>
> >
> >      <group title="Attachment">
> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > index ef41087ae3..df99f01b64 100644
> > --- a/tests/multinode-macros.at
> > +++ b/tests/multinode-macros.at
> > @@ -73,7 +73,7 @@ m_count_rows() {
> >  m_check_row_count() {
> >      local db=$(parse_db $1) table=$(parse_table $1); shift
> >      local count=$1; shift
> > -    local found=$(m_count_rows $c $db:$table "$@")
> > +    local found=$(m_count_rows $db:$table "$@")
> >      echo
> >      echo "Checking for $count rows in $db $table${1+ with $*}... found
> $found"
> >      if test "$count" != "$found"; then
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index 1e6eeb6610..9e01a29cc2 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -1033,4 +1033,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c
> 'dd bs=512 count=2 if=/dev/uran
> >  done
> >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev
> sw0p1 | grep -q 'mtu 942'])
> >
> > +# Reset back to geneve tunnels
> > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > +do
> > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > +done
> > +
> > +AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode NAT on a provider network with no localnet
> ports])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03
> 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04
> 10.0.0.4 1000::4"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1
> 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2
> 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +
> > +m_wait_for_ports_up
> > +
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> 10.0.0.4 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create the second logical switch with one port
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03
> 20.0.0.3 2000::3"
> > +
> > +# Create a logical router and attach both logical switches
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> 1000::a/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24
> 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1
> 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > +
> > +# create exteranl connection for N/S traffic
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01
> 172.20.0.100/24
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > +
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110
> 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > +
> > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > +check multinode_nbctl lsp-add public public-port1
> > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03
> 172.168.0.50"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1
> 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +# First do basic ping tests before deleting the localnet port -
> ln-public.
> > +# Once the localnet port is deleted from public ls, routing for
> 172.20.0.0/24
> > +# is centralized on ovn-gw-1.
> > +
> > +# This function checks the North-South traffic.
> > +run_ns_traffic() {
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110],
> [ignore], [ignore])
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120],
> [ignore], [ignore])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120,
> 10.0.0.3 and 20.0.0.3
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> 10.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> 20.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +}
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the localnet port by changing the type of ln-public to VIF
> port.
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > +
> > +# cr-port should not be created for public-lr0 since the option
> > +# centralize_routing=true is not yet set for lr0-public.
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Set the option - centralize_routing now.
> > +check multinode_nbctl --wait=hv set logical_router_port lr0-public
> options:centralize_routing=true
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type
> logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Re-add the localnet port
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > +
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the ln-public port this time.
> > +check multinode_nbctl --wait=hv lsp-del ln-public
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type
> logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> >  AT_CLEANUP
> > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > index a389d19886..5445fff494 100644
> > --- a/tests/ovn-northd.at
> > +++ b/tests/ovn-northd.at
> > @@ -2188,7 +2188,7 @@ match=(inport == "lrp-public" && arp.op == 1 &&
> arp.tpa == 43.43.43.4 && is_chas
> >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /*
> ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <->
> arp.spa; outport = inport; flags.loopback = 1; output;)
> >  ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE
> 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >  AT_CLEANUP
> > @@ -5524,13 +5524,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep
> "192.168.4.100" | grep "_MC_flo
> >
> >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> >
> > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1
> localnet
> >
> >  ovn-sbctl lflow-list ls1 > ls1_lflows
> >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> action=(outport = get_fdb(eth.dst); next;)
> >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> $svc_monitor_mac && (tcp || icmp || icmp6)),
> action=(handle_svc_check(inport);)
> > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport =
> "ls1-ro1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> action=(outport = "_MC_flood"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)),
> action=(outport = "_MC_flood_l2"; output;)
> > @@ -12721,3 +12722,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep
> lr_in_dnat | ovn_strip_lflows], [0], [d
> >
> >  AT_CLEANUP
> >  ])
> > +
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([NAT on a provider network with no localnet ports])
> > +AT_KEYWORDS([NAT])
> > +ovn_start
> > +
> > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lsp-add public pub-p1
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02
> 172.168.0.10/24
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > +
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3
> sw0-port1 30:54:00:00:00:03
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > +
> > +check ovn-nbctl --wait=sb sync
> > +
> > +check_flows_no_cr_port_for_public_lr0() {
> > +  # check that there is no port binding cr-public-lr0
> > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present ||
> eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:02 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && inport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast
> ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0,
> 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport =
> "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0,
> 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   ,
> match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst
> = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha =
> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport;
> flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24),
> action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24),
> action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.100 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> == 10.0.0.0/24 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> == 20.0.0.0/24 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.100 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> $svc_monitor_mac && (tcp || icmp || icmp6)),
> action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport
> = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && inport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +check_flows_cr_port_for_public_lr0() {
> > +  # check that there is port binding cr-public-lr0
> > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +  check_column chassisredirect Port_Binding type
> logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present ||
> eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:02 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast
> ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0,
> 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport =
> "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0,
> 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   ,
> match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]];
> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]];
> arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24),
> action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24),
> action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> = 1; next; )
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.100 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> == 10.0.0.0/24 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> == 20.0.0.0/24 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.100 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> $svc_monitor_mac && (tcp || icmp || icmp6)),
> action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")),
> action=(outport = "cr-public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport
> = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Remove the localnet port from public logical switch.
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > +
> > +# Check that the lflows are as expected and there is no cr port
> > +# created for "public-lr0"  when public has no localnet port
> > +# since public doesn't have the option "overlay_provider_network=true"
> > +# set.
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && inport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +
> > +# Set the option "centralize_routing=true" for lr0-public.
> > +check ovn-nbctl --wait=sb set logical_router_port lr0-public
> options:centralize_routing=true
> > +
> > +# Check that the lflows are as expected and there is cr port created
> for public-lr0.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Set the type of ln-public back to localnet
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Delete the localnet port
> > +check ovn-nbctl --wait=sb lsp-del ln-public
> > +
> > +# Check that the lflows are as expected when public has no localnet
> port.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Create multiple gateway ports.  chassisresident port should not be
> > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > +# logical switch.
> > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > +# check that there is no port binding cr-public-lr0
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && inport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.168.0.120 && inport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.110 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> 172.168.0.120 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> == 20.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CLEANUP
> > +])
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index 185ba4a21e..21d7484f9e 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -21232,10 +21232,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set
> Logical_Switch_Port rp-sw0 \
> >      type=router options:router-port=sw0 \
> >      -- lsp-set-addresses rp-sw0 router
> >
> > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24
> 2002:0:0:0:0:0:0:1/64 \
> > -    -- lrp-set-gateway-chassis sw1 hv2
> > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24
> 2002:0:0:0:0:0:0:1/64 \
> > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > -    type=router options:router-port=sw1 \
> > +    type=router options:router-port=lr0-sw1 \
> >      -- lsp-set-addresses rp-sw1 router
> >
> >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > @@ -21247,6 +21247,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> >  ovn-nbctl lsp-add sw1 sw1-p0 \
> >      -- lsp-set-addresses sw1-p0 unknown
> >
> > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > +
> >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> >
> > --
> > 2.45.2
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Numan Siddique July 25, 2024, 3:28 p.m. UTC | #3
On Thu, Jul 25, 2024 at 11:21 AM Roberto Bartzen Acosta via dev
<ovs-dev@openvswitch.org> wrote:
>
> Em qui., 25 de jul. de 2024 às 11:12, Numan Siddique <numans@ovn.org>
> escreveu:
>
> > On Tue, Jul 9, 2024 at 7:25 PM <numans@ovn.org> wrote:
> > >
> > > From: Numan Siddique <numans@ovn.org>
> > >
> > > Consider a deployment with the below logical resources:
> > >
> > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > >    port ln-public.
> > > 2. A logical router 'R'
> > > 3. Logical switch 'public' connected to R via logical switch/router port
> > >    peers (public-R and R-public).
> > > 4. R-public is distributed gateway port with its network as
> > 172.16.0.0/24
> > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > 6. And a few overlay logical switches S1, S2 to R.
> > >
> > > Any traffic from logical port - P1 of public logical switch destined to
> > > S1 or S2's logical ports goes out of the source chassis
> > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > which handles the routing.
> > >
> > > There are couple of traffic flow scenarios which doesn't work if the
> > > logical switch 'public' doesn't have a localnet port.
> > >
> > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > >    dropped in the source chassis.  The packet enters the router R's
> > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > >    the logical flow to allow traffic destined to the distributed gateway
> > >    port MAC is installed only on the gateway chassis.
> > >
> > > 2. NAT doesn't work as expected.
> > >
> > > In order to suppose this use case (of a logical switch not having a
> > > localnet port, but has a distributed gateway port and NATs), this patch
> > > supports the option 'centralize_routing', which can be configured on
> > > the distributed gateway port (R-public in the example above).
> > > If this option is set, then routing is centralized on the gateway
> > > chassis for the traffic destined to the R-public's networks
> > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > tunnelled to the gateway chassis.
> > >
> > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > logical switch port - public-R, along with cr-R-public inorder to
> > > centralize the traffic.
> > >
> > > This feature gets enabled for the distributed gateway port R-public if
> > >   - The above option is set to true in the R-public's options column.
> > >   - The logical switch 'public' doesn't have any localnet ports.
> > >   - And R-public is the only distributed gateway port of R.
> > >
> > > Distributed NAT (i.e if external_mac and router_port is set) is
> > > not supported and instead the router port mac is used for such traffic
> > > and centralized on the gateway chassis.
> > >
> > > Reported-at: https://issues.redhat.com/browse/FDP-364
> > > Signed-off-by: Numan Siddique <numans@ovn.org>
> > > ---
> > >  NEWS                      |   2 +
> > >  controller/physical.c     |   4 +
> > >  northd/northd.c           | 257 +++++++++++++++----
> > >  northd/northd.h           |   1 +
> > >  ovn-nb.xml                |  34 +++
> > >  tests/multinode-macros.at |   2 +-
> > >  tests/multinode.at        | 177 +++++++++++++
> > >  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
> > >  tests/ovn.at              |   8 +-
> > >  9 files changed, 951 insertions(+), 50 deletions(-)
> > >
> > > diff --git a/NEWS b/NEWS
> > > index 3e392ff08b..472445a188 100644
> > > --- a/NEWS
> > > +++ b/NEWS
> > > @@ -38,6 +38,8 @@ Post v24.03.0
> > >      ability to disable "VXLAN mode" to extend available tunnel IDs
> > space for
> > >      datapaths from 4095 to 16711680.  For more details see man
> > ovn-nb(5) for
> > >      mentioned option.
> > > +  - Added Overlay provider network support to a logical switch if
> > > +    the config "overlay_provider_network" is set to true.
> >
> > Please ignore this modification to NEWS.  It's wrong.  I'll fix it in
> > the next version.
> >
> > Numan
> >
> > >
> > >  OVN v24.03.0 - 01 Mar 2024
> > >  --------------------------
> > > diff --git a/controller/physical.c b/controller/physical.c
> > > index 22756810fd..e3a316989a 100644
> > > --- a/controller/physical.c
> > > +++ b/controller/physical.c
> > > @@ -1608,6 +1608,10 @@ consider_port_binding(struct ovsdb_idl_index
> > *sbrec_port_binding_by_name,
> > >                                                      ct_zones);
> > >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> > >
> > > +            /* Clear the MFF_INPORT.  Its possible that the same packet
> > may
> > > +             * go out from the same tunnel inport. */
> > > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16,
> > ofpacts_p);
> > > +
> > >              /* Resubmit to table 41. */
> > >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> > >          }
> > > diff --git a/northd/northd.c b/northd/northd.c
> > > index 6898daa00d..9b52d5a3c0 100644
> > > --- a/northd/northd.c
> > > +++ b/northd/northd.c
> > > @@ -2099,6 +2099,55 @@ parse_lsp_addrs(struct ovn_port *op)
> > >      }
> > >  }
> > >
> > > +static struct ovn_port *
> > > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > > +{
> > > +    char *redirect_name = ovn_chassis_redirect_name(
> > > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > > +
> > > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > > +        ovn_port_set_nb(crp, NULL, op->nbrp);
> > > +        ovs_list_remove(&crp->list);
> > > +        ovs_list_push_back(both_dbs, &crp->list);
> > > +    } else {
> > > +        crp = ovn_port_create(ports, redirect_name,
> > > +                              op->nbsp, op->nbrp, NULL);
> > > +        ovs_list_push_back(nb_only, &crp->list);
> > > +    }
> > > +
> > > +    crp->primary_port = op;
> > > +    op->cr_port = crp;
> > > +    crp->od = op->od;
> > > +    free(redirect_name);
> > > +
> > > +    return crp;
> > > +}
> > > +
> > > +/* Returns true if chassis resident port needs to be created for
> > > + * op's peer logical switch.  False otherwise.
> > > + *
> > > + * Chassis resident port needs to be created if the following
> > > + * conditionsd are met:
> > > + *   - op is a distributed gateway port
> > > + *   - op has the option 'centralize_routing' set to true
> > > + *   - op is the only distributed gateway port attached to its
> > > + *     router
> > > + *   - op's peer logical switch has no localnet ports.
> > > + */
> > > +static bool
> > > +peer_needs_cr_port_creation(struct ovn_port *op)
> > > +{
> > > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > > +        && !op->peer->od->n_localnet_ports) {
> > > +        return smap_get_bool(&op->nbrp->options, "centralize_routing",
> > false);
> > > +    }
> > > +
> > > +    return false;
> > > +}
> > > +
> > >  static void
> > >  join_logical_ports(const struct sbrec_port_binding_table
> > *sbrec_pb_table,
> > >                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > > @@ -2206,9 +2255,10 @@ join_logical_ports(const struct
> > sbrec_port_binding_table *sbrec_pb_table,
> > >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> > >          }
> > >      }
> > > +
> > > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> > >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> > >          ovs_assert(od->nbr);
> > > -        size_t n_allocated_l3dgw_ports = 0;
> > >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> > >              const struct nbrec_logical_router_port *nbrp
> > >                  = od->nbr->ports[i];
> > > @@ -2272,10 +2322,7 @@ join_logical_ports(const struct
> > sbrec_port_binding_table *sbrec_pb_table,
> > >                      redirect_type && !strcasecmp(redirect_type,
> > "bridged");
> > >              }
> > >
> > > -            if (op->nbrp->ha_chassis_group ||
> > > -                op->nbrp->n_gateway_chassis) {
> > > -                /* Additional "derived" ovn_port crp represents the
> > > -                 * instance of op on the gateway chassis. */
> > > +            if (op->nbrp->ha_chassis_group ||
> > op->nbrp->n_gateway_chassis) {
> > >                  const char *gw_chassis = smap_get(&op->od->nbr->options,
> > >                                                 "chassis");
> > >                  if (gw_chassis) {
> > > @@ -2284,34 +2331,9 @@ join_logical_ports(const struct
> > sbrec_port_binding_table *sbrec_pb_table,
> > >                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> > >                                   "gateway port configured on port %s "
> > >                                   "on L3 gateway router", nbrp->name);
> > > -                    continue;
> > > -                }
> > > -
> > > -                char *redirect_name =
> > > -                    ovn_chassis_redirect_name(nbrp->name);
> > > -                struct ovn_port *crp = ovn_port_find(ports,
> > redirect_name);
> > > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > > -                    ovs_list_remove(&crp->list);
> > > -                    ovs_list_push_back(both, &crp->list);
> > >                  } else {
> > > -                    crp = ovn_port_create(ports, redirect_name,
> > > -                                          NULL, nbrp, NULL);
> > > -                    ovs_list_push_back(nb_only, &crp->list);
> > > -                }
> > > -                crp->primary_port = op;
> > > -                op->cr_port = crp;
> > > -                crp->od = od;
> > > -                free(redirect_name);
> > > -
> > > -                /* Add to l3dgw_ports in od, for later use during flow
> > > -                 * creation. */
> > > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > -
> >  &n_allocated_l3dgw_ports,
> > > -                                                 sizeof
> > *od->l3dgw_ports);
> > > +                    hmapx_add(&dgps, op);
> > >                  }
> > > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > >             }
> > >          }
> > >      }
> > > @@ -2368,12 +2390,6 @@ join_logical_ports(const struct
> > sbrec_port_binding_table *sbrec_pb_table,
> > >                          arp_proxy, op->nbsp->name);
> > >                  }
> > >              }
> > > -
> > > -            /* Only used for the router type LSP whose peer is
> > l3dgw_port */
> > > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > > -                op->enable_router_port_acl = smap_get_bool(
> > > -                    &op->nbsp->options, "enable_router_port_acl",
> > false);
> > > -            }
> > >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> > >              struct ovn_port *peer = ovn_port_find(ports,
> > op->nbrp->peer);
> > >              if (peer) {
> > > @@ -2394,6 +2410,57 @@ join_logical_ports(const struct
> > sbrec_port_binding_table *sbrec_pb_table,
> > >          }
> > >      }
> > >
> > > +    struct hmapx_node *hmapx_node;
> > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > +        op = hmapx_node->data;
> > > +        od = op->od;
> > > +        ovs_assert(op->nbrp);
> > > +        ovs_assert(op->nbrp->ha_chassis_group ||
> > op->nbrp->n_gateway_chassis);
> > > +
> > > +        /* Additional "derived" ovn_port crp represents the instance of
> > op on
> > > +         * the gateway chassis. */
> > > +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> > > +        ovs_assert(crp);
> > > +
> > > +        /* Add to l3dgw_ports in od, for later use during flow
> > creation. */
> > > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > +                                        &od->n_allocated_l3dgw_ports,
> > > +                                        sizeof *od->l3dgw_ports);
> > > +        }
> > > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > > +
> > > +        if (op->peer && op->peer->nbsp) {
> > > +            /* Only used for the router type LSP whose peer is
> > l3dgw_port */
> > > +            op->peer->enable_router_port_acl = smap_get_bool(
> > > +                    &op->peer->nbsp->options, "enable_router_port_acl",
> > false);
> > > +        }
> > > +    }
> > > +
> > > +
> > > +    /* Create chassisresident port for the distributed gateway port's
> > (DGP)
> > > +     * peer if
> > > +     *  - DGP's router has only one DGP and
> > > +     *  - Its peer is a logical switch port and
> > > +     *  - It's peer's logical switch has no localnet ports and
> > > +     *  - option 'centralize_routing' is set to true for the DGP.
> > > +     *
> > > +     * This is required to support
> > > +     *   - NAT via geneve (for the overlay provider networks) and
> > > +     *   - to centralize routing on the gateway chassis for the traffic
> > > +     *     destined to the DGP's networks.
> > > +     *
> > > +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> > > +     * of a logical router.
> > > +     * */
> > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > +        op = hmapx_node->data;
> > > +        if (peer_needs_cr_port_creation(op)) {
> > > +            create_cr_port(op->peer, ports, both, nb_only);
> > > +        }
> > > +    }
> > > +    hmapx_destroy(&dgps);
> > > +
> > >      /* Wait until all ports have been connected to add to IPAM since
> > >       * it relies on proper peers to be set
> > >       */
> > > @@ -3176,16 +3243,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn
> > *ovnsb_txn,
> > >               * type "l3gateway". */
> > >              if (chassis) {
> > >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > > +            } else if (is_cr_port(op)) {
> > > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > > +                ovs_assert(op->primary_port->peer);
> > > +                ovs_assert(op->primary_port->peer->cr_port);
> > > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > > +                sbrec_port_binding_set_ha_chassis_group(
> > > +                    op->sb,
> > > +
> > op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > > +
> > >              } else {
> > >                  sbrec_port_binding_set_type(op->sb, "patch");
> > >              }
> > >
> > >              const char *router_port = smap_get(&op->nbsp->options,
> > >                                                 "router-port");
> > > -            if (router_port || chassis) {
> > > +            if (router_port || chassis || is_cr_port(op)) {
> > >                  struct smap new;
> > >                  smap_init(&new);
> > > -                if (router_port) {
> > > +
> > > +                if (is_cr_port(op)) {
> > > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > > +                } else if (router_port) {
> > >                      smap_add(&new, "peer", router_port);
> > >                  }
> > >                  if (chassis) {
> > > @@ -8191,9 +8270,27 @@ build_lswitch_rport_arp_req_flow(
> > >      struct lflow_ref *lflow_ref)
> > >  {
> > >      struct ds match   = DS_EMPTY_INITIALIZER;
> > > +    struct ds m       = DS_EMPTY_INITIALIZER;
> > >      struct ds actions = DS_EMPTY_INITIALIZER;
> > >
> > > -    arp_nd_ns_match(ips, addr_family, &match);
> > > +    arp_nd_ns_match(ips, addr_family, &m);
> > > +    ds_clone(&match, &m);
> > > +
> > > +    bool has_cr_port = patch_op->cr_port;
> > > +
> > > +    /* If the patch_op has a chassis resident port, it means
> > > +     *    - its peer is a distributed gateway port (DGP) and
> > > +     *    - routing is centralized for the DGP's networks on
> > > +     *      the configured gateway chassis.
> > > +     *
> > > +     * If that's the case, make sure that the packets destined to
> > > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > > +     * */
> > > +
> > > +    if (has_cr_port) {
> > > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > > +                      patch_op->cr_port->json_key);
> > > +    }
> > >
> > >      /* Send a the packet to the router pipeline.  If the switch has
> > non-router
> > >       * ports then flood it there as well.
> > > @@ -8215,6 +8312,31 @@ build_lswitch_rport_arp_req_flow(
> > >                                  lflow_ref);
> > >      }
> > >
> > > +    if (has_cr_port) {
> > > +        ds_clear(&match);
> > > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)",
> > ds_cstr(&m),
> > > +                      patch_op->cr_port->json_key);
> > > +        ds_clear(&actions);
> > > +        if (od->n_router_ports != od->nbs->n_ports) {
> > > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > > +                                    "outport = \""MC_FLOOD_L2"\";
> > output;",
> > > +                          patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority, ds_cstr(&match),
> > > +                                    ds_cstr(&actions), stage_hint,
> > > +                                    lflow_ref);
> > > +        } else {
> > > +            ds_put_format(&actions, "outport = %s; output;",
> > > +                          patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority, ds_cstr(&match),
> > > +                                    ds_cstr(&actions),
> > > +                                    stage_hint,
> > > +                                    lflow_ref);
> > > +        }
> > > +    }
> > > +
> > > +    ds_destroy(&m);
> > >      ds_destroy(&match);
> > >      ds_destroy(&actions);
> > >  }
> > > @@ -9585,7 +9707,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> > *op,
> > >                                  struct ds *actions, struct ds *match)
> > >  {
> > >      ovs_assert(op->nbsp);
> > > -    if (lsp_is_external(op->nbsp)) {
> > > +
> > > +    /* Note: A switch port can also have a chassis resident derived
> > port.
> > > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > > +     * adding unicast lookup flows for this port. */
> > > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> > >          return;
> > >      }
> > >
> > > @@ -9603,8 +9729,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> > *op,
> > >                             "outport = \""MC_UNKNOWN "\"; output;"
> > >                           : "outport = %s; output;")
> > >                           : debug_drop_action();
> > > -    ds_clear(actions);
> > > -    ds_put_format(actions, action, op->json_key);
> > >
> > >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> > >          /* For ports connected to logical routers add flows to bypass
> > the
> > > @@ -9651,14 +9775,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> > *op,
> > >              if (add_chassis_resident_check) {
> > >                  ds_put_format(match, " && is_chassis_resident(%s)",
> > json_key);
> > >              }
> > > +        } else if (op->cr_port) {
> > > +            /* If the op has a chassis resident port, it means
> > > +             *   - its peer is a distributed gateway port (DGP) and
> > > +             *   - routing is centralized for the DGP's networks on
> > > +             *     the configured gateway chassis.
> > > +             *
> > > +             * If that's the case, make sure that the packets destined
> > to
> > > +             * the DGP's MAC are sent to the chassis where the DGP
> > resides.
> > > +             * */
> > > +            ds_clear(actions);
> > > +            ds_put_format(actions, action, op->cr_port->json_key);
> > > +
> > > +            struct ds m = DS_EMPTY_INITIALIZER;
> > > +            ds_put_format(&m, "eth.dst == %s &&
> > !is_chassis_resident(%s)",
> > > +                          op->peer->lrp_networks.ea_s,
> > > +                          op->cr_port->json_key);
> > > +
> > > +            ovn_lflow_add_with_hint(lflows, op->od,
> > > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > > +                                    ds_cstr(&m), ds_cstr(actions),
> > > +                                    &op->nbsp->header_,
> > > +                                    op->lflow_ref);
> > > +            ds_destroy(&m);
> > > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > > +                          op->cr_port->json_key);
> > >          }
> > >
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >          ovn_lflow_add_with_hint(lflows, op->od,
> > >                                  S_SWITCH_IN_L2_LKUP, 50,
> > >                                  ds_cstr(match), ds_cstr(actions),
> > >                                  &op->nbsp->header_,
> > >                                  op->lflow_ref);
> > >      } else {
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> > >              ds_clear(match);
> > >              ds_put_format(match, "eth.dst == %s",
> > op->lsp_addrs[i].ea_s);
> > > @@ -11772,6 +11925,14 @@ build_lrouter_port_nat_arp_nd_flow(struct
> > ovn_port *op,
> > >          return;
> > >      }
> > >
> > > +    if (op->peer && op->peer->cr_port) {
> > > +        /* We don't add the below flows if the router port's peer has
> > > +         * a chassisresident port.  That's because routing is
> > centralized on
> > > +         * the gateway chassis for the router port networks/subnets.
> > > +         */
> > > +        return;
> > > +    }
> > > +
> > >      /* Mac address to use when replying to ARP/NS. */
> > >      const char *mac_s = REG_INPORT_ETH_ADDR;
> > >      struct eth_addr mac;
> > > @@ -15158,6 +15319,16 @@ lrouter_check_nat_entry(const struct
> > ovn_datapath *od,
> > >      /* For distributed router NAT, determine whether this NAT rule
> > >       * satisfies the conditions for distributed NAT processing. */
> > >      *distributed = false;
> > > +
> > > +    /* NAT cannnot be distributed if the DGP's peer
> > > +     * has a chassisresident port (as the routing is centralized
> > > +     * on the gateway chassis for the DGP's networks/subnets.)
> > > +     */
> >
>
> Hi Numan,
>
> I assume that this implementation as a whole will not affect the stateless
> use case in which the logical_router has multiple DGPs for different
> ovn-chassis (chassisresident port), and with a stateless NAT rule so that
> the traffic is decentralized via ECMP, right?

Hi Roberto,

In order for this feature to kick in,  the option 'centralize_routing
should be set on
the dgw router port and the router should only have one DGP.  Your use case
will not be affected since you'd be having multiple DGPs.

Thanks
Numan

>
> I mean, if I have a dgw_port peer for each chassis and configure a
> stateless NAT on the router, I will continue creating the static DNAT and
> SNAT path regardless of the port/chassis that the traffic passes through,
> right? otherwise, this will make it impossible to balance traffic across
> different chassis for the same logical_router.
>
> Kind regards,
> Roberto
>
>
> > > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > > +        return 0;
> > > +    }
> > > +
> > >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> > >          nat->logical_port && nat->external_mac) {
> > >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > > diff --git a/northd/northd.h b/northd/northd.h
> > > index d4a8d75abc..d7c9655916 100644
> > > --- a/northd/northd.h
> > > +++ b/northd/northd.h
> > > @@ -325,6 +325,7 @@ struct ovn_datapath {
> > >       * will be NULL. */
> > >      struct ovn_port **l3dgw_ports;
> > >      size_t n_l3dgw_ports;
> > > +    size_t n_allocated_l3dgw_ports;
> > >
> > >      /* router datapath has a logical port with redirect-type set to
> > bridged. */
> > >      bool redirect_bridged;
> > > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > > index 9552534f6d..794e2fb961 100644
> > > --- a/ovn-nb.xml
> > > +++ b/ovn-nb.xml
> > > @@ -3451,6 +3451,40 @@ or
> > >            <ref column="options" key="gateway_mtu"/> option.
> > >          </p>
> > >        </column>
> > > +
> > > +      <column name="options" key="centralize_routing"
> > > +              type='{"type": "boolean"}'>
> > > +        <p>
> > > +          This option is applicable only if the router port is a
> > > +          distributed gateway port i.e if the <ref
> > table="Logical_Router_Port"
> > > +          column="ha_chassis_group"/> column or
> > > +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> > > +          is set.
> > > +        </p>
> > > +
> > > +        <p>
> > > +          If set to <code>true</code>, routing for the router port's
> > > +          networks (set in the column <ref table="Logical_Router_Port"
> > > +          column="networks"/>) is centralized on the gateway chassis
> > > +          which claims this distributed gateway port.
> > > +        </p>
> > > +
> > > +        <p>
> > > +          Additionally for this option to take effect, below conditions
> > > +          must be met:
> > > +        </p>
> > > +
> > > +        <ul>
> > > +          <li>
> > > +            The Logical router has only one distributed gateway port.
> > > +          </li>
> > > +
> > > +          <li>
> > > +            The router port's peer logical switch has no localnet ports.
> > > +          </li>
> > > +
> > > +        </ul>
> > > +      </column>
> > >      </group>
> > >
> > >      <group title="Attachment">
> > > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > > index ef41087ae3..df99f01b64 100644
> > > --- a/tests/multinode-macros.at
> > > +++ b/tests/multinode-macros.at
> > > @@ -73,7 +73,7 @@ m_count_rows() {
> > >  m_check_row_count() {
> > >      local db=$(parse_db $1) table=$(parse_table $1); shift
> > >      local count=$1; shift
> > > -    local found=$(m_count_rows $c $db:$table "$@")
> > > +    local found=$(m_count_rows $db:$table "$@")
> > >      echo
> > >      echo "Checking for $count rows in $db $table${1+ with $*}... found
> > $found"
> > >      if test "$count" != "$found"; then
> > > diff --git a/tests/multinode.at b/tests/multinode.at
> > > index 1e6eeb6610..9e01a29cc2 100644
> > > --- a/tests/multinode.at
> > > +++ b/tests/multinode.at
> > > @@ -1033,4 +1033,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c
> > 'dd bs=512 count=2 if=/dev/uran
> > >  done
> > >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev
> > sw0p1 | grep -q 'mtu 942'])
> > >
> > > +# Reset back to geneve tunnels
> > > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > > +do
> > > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > > +done
> > > +
> > > +AT_CLEANUP
> > > +
> > > +AT_SETUP([ovn multinode NAT on a provider network with no localnet
> > ports])
> > > +
> > > +# Check that ovn-fake-multinode setup is up and running
> > > +check_fake_multinode_setup
> > > +
> > > +# Delete the multinode NB and OVS resources before starting the test.
> > > +cleanup_multinode_resources
> > > +
> > > +check multinode_nbctl ls-add sw0
> > > +check multinode_nbctl lsp-add sw0 sw0-port1
> > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03
> > 10.0.0.3 1000::3"
> > > +check multinode_nbctl lsp-add sw0 sw0-port2
> > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04
> > 10.0.0.4 1000::4"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1
> > 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2
> > 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > > +
> > > +m_wait_for_ports_up
> > > +
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > 10.0.0.4 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# Create the second logical switch with one port
> > > +check multinode_nbctl ls-add sw1
> > > +check multinode_nbctl lsp-add sw1 sw1-port1
> > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03
> > 20.0.0.3 2000::3"
> > > +
> > > +# Create a logical router and attach both logical switches
> > > +check multinode_nbctl lr-add lr0
> > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > 1000::a/64
> > > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24
> > 2000::a/64
> > > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1
> > 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > > +
> > > +# create exteranl connection for N/S traffic
> > > +check multinode_nbctl ls-add public
> > > +check multinode_nbctl lsp-add public ln-public
> > > +check multinode_nbctl lsp-set-type ln-public localnet
> > > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01
> > 172.20.0.100/24
> > > +check multinode_nbctl lsp-add public public-lr0
> > > +check multinode_nbctl lsp-set-type public-lr0 router
> > > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > > +
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110
> > 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > > +
> > > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > > +check multinode_nbctl lsp-add public public-port1
> > > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03
> > 172.168.0.50"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1
> > 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +
> > > +# First do basic ping tests before deleting the localnet port -
> > ln-public.
> > > +# Once the localnet port is deleted from public ls, routing for
> > 172.20.0.0/24
> > > +# is centralized on ovn-gw-1.
> > > +
> > > +# This function checks the North-South traffic.
> > > +run_ns_traffic() {
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110],
> > [ignore], [ignore])
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120],
> > [ignore], [ignore])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120,
> > 10.0.0.3 and 20.0.0.3
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> > 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> > 10.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2
> > 20.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +}
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the localnet port by changing the type of ln-public to VIF
> > port.
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > > +
> > > +# cr-port should not be created for public-lr0 since the option
> > > +# centralize_routing=true is not yet set for lr0-public.
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Set the option - centralize_routing now.
> > > +check multinode_nbctl --wait=hv set logical_router_port lr0-public
> > options:centralize_routing=true
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type
> > logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Re-add the localnet port
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > > +
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the ln-public port this time.
> > > +check multinode_nbctl --wait=hv lsp-del ln-public
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type
> > logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > >  AT_CLEANUP
> > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > > index a389d19886..5445fff494 100644
> > > --- a/tests/ovn-northd.at
> > > +++ b/tests/ovn-northd.at
> > > @@ -2188,7 +2188,7 @@ match=(inport == "lrp-public" && arp.op == 1 &&
> > arp.tpa == 43.43.43.4 && is_chas
> > >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /*
> > ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <->
> > arp.spa; outport = inport; flags.loopback = 1; output;)
> > >  ])
> > >
> > > -# xreg0[0..47] isn't used anywhere else.
> > > +# xreg0[[0..47]] isn't used anywhere else.
> > >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE
> > 'lr_in_admission|lr_in_ip_input'], [1], [])
> > >
> > >  AT_CLEANUP
> > > @@ -5524,13 +5524,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep
> > "192.168.4.100" | grep "_MC_flo
> > >
> > >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> > >
> > > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1
> > localnet
> > >
> > >  ovn-sbctl lflow-list ls1 > ls1_lflows
> > >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> > >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > action=(outport = get_fdb(eth.dst); next;)
> > >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > action=(handle_svc_check(inport);)
> > > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport =
> > "ls1-ro1"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > action=(outport = "_MC_flood"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)),
> > action=(outport = "_MC_flood_l2"; output;)
> > > @@ -12721,3 +12722,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep
> > lr_in_dnat | ovn_strip_lflows], [0], [d
> > >
> > >  AT_CLEANUP
> > >  ])
> > > +
> > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > +AT_SETUP([NAT on a provider network with no localnet ports])
> > > +AT_KEYWORDS([NAT])
> > > +ovn_start
> > > +
> > > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > > +check ovn-nbctl lsp-add sw0 sw0-port1
> > > +check ovn-nbctl lr-add lr0
> > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > > +check ovn-nbctl ls-add public
> > > +check ovn-nbctl lsp-add public pub-p1
> > > +
> > > +# localnet port
> > > +check ovn-nbctl lsp-add public ln-public
> > > +check ovn-nbctl lsp-set-type ln-public localnet
> > > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02
> > 172.168.0.10/24
> > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > > +
> > > +check ovn-nbctl lsp-add public public-lr0
> > > +check ovn-nbctl lsp-set-type public-lr0 router
> > > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3
> > sw0-port1 30:54:00:00:00:03
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > > +
> > > +check ovn-nbctl --wait=sb sync
> > > +
> > > +check_flows_no_cr_port_for_public_lr0() {
> > > +  # check that there is no port binding cr-public-lr0
> > > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> > action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present ||
> > eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> > !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> > action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:02 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast
> > ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> > 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> > "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0,
> > 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport =
> > "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> > ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> > ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0,
> > 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> > ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   ,
> > match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> > nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> > 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst
> > = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha =
> > arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport;
> > flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> > xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> > inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24),
> > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> > outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24),
> > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> > outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> > 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.100 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> > action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> > action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> > action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> > ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> > "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> > action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> > action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> > == 10.0.0.0/24 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> > == 20.0.0.0/24 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.100 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport
> > = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> > 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +check_flows_cr_port_for_public_lr0() {
> > > +  # check that there is port binding cr-public-lr0
> > > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +  check_column chassisredirect Port_Binding type
> > logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> > action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present ||
> > eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> > !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> > action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:02 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast
> > ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> > 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 =
> > 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> > "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0,
> > 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport =
> > "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> > ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0;
> > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> > ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0,
> > 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> > ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   ,
> > match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> > nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> > action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> > 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]];
> > arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]];
> > arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> > xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> > inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24),
> > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> > outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24),
> > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa;
> > outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> > nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback
> > = 1; next; )
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.100 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> > action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> > action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> > action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> > ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> > "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> > action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> > action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> > == 10.0.0.0/24 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src
> > == 20.0.0.0/24 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> > action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.100 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> > [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")),
> > action=(outport = "cr-public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport
> > = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Remove the localnet port from public logical switch.
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > > +
> > > +# Check that the lflows are as expected and there is no cr port
> > > +# created for "public-lr0"  when public has no localnet port
> > > +# since public doesn't have the option "overlay_provider_network=true"
> > > +# set.
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> > 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +
> > > +# Set the option "centralize_routing=true" for lr0-public.
> > > +check ovn-nbctl --wait=sb set logical_router_port lr0-public
> > options:centralize_routing=true
> > > +
> > > +# Check that the lflows are as expected and there is cr port created
> > for public-lr0.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Set the type of ln-public back to localnet
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Delete the localnet port
> > > +check ovn-nbctl --wait=sb lsp-del ln-public
> > > +
> > > +# Check that the lflows are as expected when public has no localnet
> > port.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Create multiple gateway ports.  chassisresident port should not be
> > > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > > +# logical switch.
> > > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > > +# check that there is no port binding cr-public-lr0
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")),
> > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 &&
> > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback =
> > 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src =
> > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1;
> > output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> > == 172.168.0.120 && inport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.110 && outport == "lr0-public" &&
> > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > 172.168.0.120 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport =
> > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1;
> > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7
> > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??);
> > };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")
> > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src
> > == 20.0.0.3 && outport == "lr0-public" &&
> > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e
> > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport =
> > "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 ||
> > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0
> > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CLEANUP
> > > +])
> > > diff --git a/tests/ovn.at b/tests/ovn.at
> > > index 185ba4a21e..21d7484f9e 100644
> > > --- a/tests/ovn.at
> > > +++ b/tests/ovn.at
> > > @@ -21232,10 +21232,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set
> > Logical_Switch_Port rp-sw0 \
> > >      type=router options:router-port=sw0 \
> > >      -- lsp-set-addresses rp-sw0 router
> > >
> > > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24
> > 2002:0:0:0:0:0:0:1/64 \
> > > -    -- lrp-set-gateway-chassis sw1 hv2
> > > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24
> > 2002:0:0:0:0:0:0:1/64 \
> > > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> > >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > > -    type=router options:router-port=sw1 \
> > > +    type=router options:router-port=lr0-sw1 \
> > >      -- lsp-set-addresses rp-sw1 router
> > >
> > >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > > @@ -21247,6 +21247,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> > >  ovn-nbctl lsp-add sw1 sw1-p0 \
> > >      -- lsp-set-addresses sw1-p0 unknown
> > >
> > > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > > +
> > >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> > >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> > >
> > > --
> > > 2.45.2
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
>
> --
>
>
>
>
> _‘Esta mensagem é direcionada apenas para os endereços constantes no
> cabeçalho inicial. Se você não está listado nos endereços constantes no
> cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão
> imediatamente anuladas e proibidas’._
>
>
> * **‘Apesar do Magazine Luiza tomar
> todas as precauções razoáveis para assegurar que nenhum vírus esteja
> presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por
> quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
>
>
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Roberto Bartzen Acosta July 25, 2024, 4:42 p.m. UTC | #4
Em qui., 25 de jul. de 2024 às 12:28, Numan Siddique <numans@ovn.org>
escreveu:

> On Thu, Jul 25, 2024 at 11:21 AM Roberto Bartzen Acosta via dev
> <ovs-dev@openvswitch.org> wrote:
> >
> > Em qui., 25 de jul. de 2024 às 11:12, Numan Siddique <numans@ovn.org>
> > escreveu:
> >
> > > On Tue, Jul 9, 2024 at 7:25 PM <numans@ovn.org> wrote:
> > > >
> > > > From: Numan Siddique <numans@ovn.org>
> > > >
> > > > Consider a deployment with the below logical resources:
> > > >
> > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > >    port ln-public.
> > > > 2. A logical router 'R'
> > > > 3. Logical switch 'public' connected to R via logical switch/router
> port
> > > >    peers (public-R and R-public).
> > > > 4. R-public is distributed gateway port with its network as
> > > 172.16.0.0/24
> > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > 6. And a few overlay logical switches S1, S2 to R.
> > > >
> > > > Any traffic from logical port - P1 of public logical switch destined
> to
> > > > S1 or S2's logical ports goes out of the source chassis
> > > > (where P1 resides) via the localnet port and reaches the gateway
> chassis
> > > > which handles the routing.
> > > >
> > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > logical switch 'public' doesn't have a localnet port.
> > > >
> > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > >    dropped in the source chassis.  The packet enters the router R's
> > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > >    the logical flow to allow traffic destined to the distributed
> gateway
> > > >    port MAC is installed only on the gateway chassis.
> > > >
> > > > 2. NAT doesn't work as expected.
> > > >
> > > > In order to suppose this use case (of a logical switch not having a
> > > > localnet port, but has a distributed gateway port and NATs), this
> patch
> > > > supports the option 'centralize_routing', which can be configured on
> > > > the distributed gateway port (R-public in the example above).
> > > > If this option is set, then routing is centralized on the gateway
> > > > chassis for the traffic destined to the R-public's networks
> > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > tunnelled to the gateway chassis.
> > > >
> > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > centralize the traffic.
> > > >
> > > > This feature gets enabled for the distributed gateway port R-public
> if
> > > >   - The above option is set to true in the R-public's options column.
> > > >   - The logical switch 'public' doesn't have any localnet ports.
> > > >   - And R-public is the only distributed gateway port of R.
> > > >
> > > > Distributed NAT (i.e if external_mac and router_port is set) is
> > > > not supported and instead the router port mac is used for such
> traffic
> > > > and centralized on the gateway chassis.
> > > >
> > > > Reported-at: https://issues.redhat.com/browse/FDP-364
> > > > Signed-off-by: Numan Siddique <numans@ovn.org>
> > > > ---
> > > >  NEWS                      |   2 +
> > > >  controller/physical.c     |   4 +
> > > >  northd/northd.c           | 257 +++++++++++++++----
> > > >  northd/northd.h           |   1 +
> > > >  ovn-nb.xml                |  34 +++
> > > >  tests/multinode-macros.at |   2 +-
> > > >  tests/multinode.at        | 177 +++++++++++++
> > > >  tests/ovn-northd.at       | 516
> +++++++++++++++++++++++++++++++++++++-
> > > >  tests/ovn.at              |   8 +-
> > > >  9 files changed, 951 insertions(+), 50 deletions(-)
> > > >
> > > > diff --git a/NEWS b/NEWS
> > > > index 3e392ff08b..472445a188 100644
> > > > --- a/NEWS
> > > > +++ b/NEWS
> > > > @@ -38,6 +38,8 @@ Post v24.03.0
> > > >      ability to disable "VXLAN mode" to extend available tunnel IDs
> > > space for
> > > >      datapaths from 4095 to 16711680.  For more details see man
> > > ovn-nb(5) for
> > > >      mentioned option.
> > > > +  - Added Overlay provider network support to a logical switch if
> > > > +    the config "overlay_provider_network" is set to true.
> > >
> > > Please ignore this modification to NEWS.  It's wrong.  I'll fix it in
> > > the next version.
> > >
> > > Numan
> > >
> > > >
> > > >  OVN v24.03.0 - 01 Mar 2024
> > > >  --------------------------
> > > > diff --git a/controller/physical.c b/controller/physical.c
> > > > index 22756810fd..e3a316989a 100644
> > > > --- a/controller/physical.c
> > > > +++ b/controller/physical.c
> > > > @@ -1608,6 +1608,10 @@ consider_port_binding(struct ovsdb_idl_index
> > > *sbrec_port_binding_by_name,
> > > >                                                      ct_zones);
> > > >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> > > >
> > > > +            /* Clear the MFF_INPORT.  Its possible that the same
> packet
> > > may
> > > > +             * go out from the same tunnel inport. */
> > > > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16,
> > > ofpacts_p);
> > > > +
> > > >              /* Resubmit to table 41. */
> > > >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> > > >          }
> > > > diff --git a/northd/northd.c b/northd/northd.c
> > > > index 6898daa00d..9b52d5a3c0 100644
> > > > --- a/northd/northd.c
> > > > +++ b/northd/northd.c
> > > > @@ -2099,6 +2099,55 @@ parse_lsp_addrs(struct ovn_port *op)
> > > >      }
> > > >  }
> > > >
> > > > +static struct ovn_port *
> > > > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > > > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > > > +{
> > > > +    char *redirect_name = ovn_chassis_redirect_name(
> > > > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > > > +
> > > > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > > > +        ovn_port_set_nb(crp, NULL, op->nbrp);
> > > > +        ovs_list_remove(&crp->list);
> > > > +        ovs_list_push_back(both_dbs, &crp->list);
> > > > +    } else {
> > > > +        crp = ovn_port_create(ports, redirect_name,
> > > > +                              op->nbsp, op->nbrp, NULL);
> > > > +        ovs_list_push_back(nb_only, &crp->list);
> > > > +    }
> > > > +
> > > > +    crp->primary_port = op;
> > > > +    op->cr_port = crp;
> > > > +    crp->od = op->od;
> > > > +    free(redirect_name);
> > > > +
> > > > +    return crp;
> > > > +}
> > > > +
> > > > +/* Returns true if chassis resident port needs to be created for
> > > > + * op's peer logical switch.  False otherwise.
> > > > + *
> > > > + * Chassis resident port needs to be created if the following
> > > > + * conditionsd are met:
> > > > + *   - op is a distributed gateway port
> > > > + *   - op has the option 'centralize_routing' set to true
> > > > + *   - op is the only distributed gateway port attached to its
> > > > + *     router
> > > > + *   - op's peer logical switch has no localnet ports.
> > > > + */
> > > > +static bool
> > > > +peer_needs_cr_port_creation(struct ovn_port *op)
> > > > +{
> > > > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > > > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > > > +        && !op->peer->od->n_localnet_ports) {
> > > > +        return smap_get_bool(&op->nbrp->options,
> "centralize_routing",
> > > false);
> > > > +    }
> > > > +
> > > > +    return false;
> > > > +}
> > > > +
> > > >  static void
> > > >  join_logical_ports(const struct sbrec_port_binding_table
> > > *sbrec_pb_table,
> > > >                     struct hmap *ls_datapaths, struct hmap
> *lr_datapaths,
> > > > @@ -2206,9 +2255,10 @@ join_logical_ports(const struct
> > > sbrec_port_binding_table *sbrec_pb_table,
> > > >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> > > >          }
> > > >      }
> > > > +
> > > > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> > > >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> > > >          ovs_assert(od->nbr);
> > > > -        size_t n_allocated_l3dgw_ports = 0;
> > > >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> > > >              const struct nbrec_logical_router_port *nbrp
> > > >                  = od->nbr->ports[i];
> > > > @@ -2272,10 +2322,7 @@ join_logical_ports(const struct
> > > sbrec_port_binding_table *sbrec_pb_table,
> > > >                      redirect_type && !strcasecmp(redirect_type,
> > > "bridged");
> > > >              }
> > > >
> > > > -            if (op->nbrp->ha_chassis_group ||
> > > > -                op->nbrp->n_gateway_chassis) {
> > > > -                /* Additional "derived" ovn_port crp represents the
> > > > -                 * instance of op on the gateway chassis. */
> > > > +            if (op->nbrp->ha_chassis_group ||
> > > op->nbrp->n_gateway_chassis) {
> > > >                  const char *gw_chassis =
> smap_get(&op->od->nbr->options,
> > > >                                                 "chassis");
> > > >                  if (gw_chassis) {
> > > > @@ -2284,34 +2331,9 @@ join_logical_ports(const struct
> > > sbrec_port_binding_table *sbrec_pb_table,
> > > >                      VLOG_WARN_RL(&rl, "Bad configuration:
> distributed "
> > > >                                   "gateway port configured on port
> %s "
> > > >                                   "on L3 gateway router",
> nbrp->name);
> > > > -                    continue;
> > > > -                }
> > > > -
> > > > -                char *redirect_name =
> > > > -                    ovn_chassis_redirect_name(nbrp->name);
> > > > -                struct ovn_port *crp = ovn_port_find(ports,
> > > redirect_name);
> > > > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > > > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > > > -                    ovs_list_remove(&crp->list);
> > > > -                    ovs_list_push_back(both, &crp->list);
> > > >                  } else {
> > > > -                    crp = ovn_port_create(ports, redirect_name,
> > > > -                                          NULL, nbrp, NULL);
> > > > -                    ovs_list_push_back(nb_only, &crp->list);
> > > > -                }
> > > > -                crp->primary_port = op;
> > > > -                op->cr_port = crp;
> > > > -                crp->od = od;
> > > > -                free(redirect_name);
> > > > -
> > > > -                /* Add to l3dgw_ports in od, for later use during
> flow
> > > > -                 * creation. */
> > > > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > > > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > > -
> > >  &n_allocated_l3dgw_ports,
> > > > -                                                 sizeof
> > > *od->l3dgw_ports);
> > > > +                    hmapx_add(&dgps, op);
> > > >                  }
> > > > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > > >             }
> > > >          }
> > > >      }
> > > > @@ -2368,12 +2390,6 @@ join_logical_ports(const struct
> > > sbrec_port_binding_table *sbrec_pb_table,
> > > >                          arp_proxy, op->nbsp->name);
> > > >                  }
> > > >              }
> > > > -
> > > > -            /* Only used for the router type LSP whose peer is
> > > l3dgw_port */
> > > > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > > > -                op->enable_router_port_acl = smap_get_bool(
> > > > -                    &op->nbsp->options, "enable_router_port_acl",
> > > false);
> > > > -            }
> > > >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> > > >              struct ovn_port *peer = ovn_port_find(ports,
> > > op->nbrp->peer);
> > > >              if (peer) {
> > > > @@ -2394,6 +2410,57 @@ join_logical_ports(const struct
> > > sbrec_port_binding_table *sbrec_pb_table,
> > > >          }
> > > >      }
> > > >
> > > > +    struct hmapx_node *hmapx_node;
> > > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > > +        op = hmapx_node->data;
> > > > +        od = op->od;
> > > > +        ovs_assert(op->nbrp);
> > > > +        ovs_assert(op->nbrp->ha_chassis_group ||
> > > op->nbrp->n_gateway_chassis);
> > > > +
> > > > +        /* Additional "derived" ovn_port crp represents the
> instance of
> > > op on
> > > > +         * the gateway chassis. */
> > > > +        struct ovn_port *crp = create_cr_port(op, ports, both,
> nb_only);
> > > > +        ovs_assert(crp);
> > > > +
> > > > +        /* Add to l3dgw_ports in od, for later use during flow
> > > creation. */
> > > > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > > > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > > +
> &od->n_allocated_l3dgw_ports,
> > > > +                                        sizeof *od->l3dgw_ports);
> > > > +        }
> > > > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > > > +
> > > > +        if (op->peer && op->peer->nbsp) {
> > > > +            /* Only used for the router type LSP whose peer is
> > > l3dgw_port */
> > > > +            op->peer->enable_router_port_acl = smap_get_bool(
> > > > +                    &op->peer->nbsp->options,
> "enable_router_port_acl",
> > > false);
> > > > +        }
> > > > +    }
> > > > +
> > > > +
> > > > +    /* Create chassisresident port for the distributed gateway
> port's
> > > (DGP)
> > > > +     * peer if
> > > > +     *  - DGP's router has only one DGP and
> > > > +     *  - Its peer is a logical switch port and
> > > > +     *  - It's peer's logical switch has no localnet ports and
> > > > +     *  - option 'centralize_routing' is set to true for the DGP.
> > > > +     *
> > > > +     * This is required to support
> > > > +     *   - NAT via geneve (for the overlay provider networks) and
> > > > +     *   - to centralize routing on the gateway chassis for the
> traffic
> > > > +     *     destined to the DGP's networks.
> > > > +     *
> > > > +     * Future enhancement: Support 'centralizerouting' for all the
> DGP's
> > > > +     * of a logical router.
> > > > +     * */
> > > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > > +        op = hmapx_node->data;
> > > > +        if (peer_needs_cr_port_creation(op)) {
> > > > +            create_cr_port(op->peer, ports, both, nb_only);
> > > > +        }
> > > > +    }
> > > > +    hmapx_destroy(&dgps);
> > > > +
> > > >      /* Wait until all ports have been connected to add to IPAM since
> > > >       * it relies on proper peers to be set
> > > >       */
> > > > @@ -3176,16 +3243,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn
> > > *ovnsb_txn,
> > > >               * type "l3gateway". */
> > > >              if (chassis) {
> > > >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > > > +            } else if (is_cr_port(op)) {
> > > > +                sbrec_port_binding_set_type(op->sb,
> "chassisredirect");
> > > > +                ovs_assert(op->primary_port->peer);
> > > > +                ovs_assert(op->primary_port->peer->cr_port);
> > > > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > > > +                sbrec_port_binding_set_ha_chassis_group(
> > > > +                    op->sb,
> > > > +
> > > op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > > > +
> > > >              } else {
> > > >                  sbrec_port_binding_set_type(op->sb, "patch");
> > > >              }
> > > >
> > > >              const char *router_port = smap_get(&op->nbsp->options,
> > > >                                                 "router-port");
> > > > -            if (router_port || chassis) {
> > > > +            if (router_port || chassis || is_cr_port(op)) {
> > > >                  struct smap new;
> > > >                  smap_init(&new);
> > > > -                if (router_port) {
> > > > +
> > > > +                if (is_cr_port(op)) {
> > > > +                    smap_add(&new, "distributed-port",
> op->nbsp->name);
> > > > +                } else if (router_port) {
> > > >                      smap_add(&new, "peer", router_port);
> > > >                  }
> > > >                  if (chassis) {
> > > > @@ -8191,9 +8270,27 @@ build_lswitch_rport_arp_req_flow(
> > > >      struct lflow_ref *lflow_ref)
> > > >  {
> > > >      struct ds match   = DS_EMPTY_INITIALIZER;
> > > > +    struct ds m       = DS_EMPTY_INITIALIZER;
> > > >      struct ds actions = DS_EMPTY_INITIALIZER;
> > > >
> > > > -    arp_nd_ns_match(ips, addr_family, &match);
> > > > +    arp_nd_ns_match(ips, addr_family, &m);
> > > > +    ds_clone(&match, &m);
> > > > +
> > > > +    bool has_cr_port = patch_op->cr_port;
> > > > +
> > > > +    /* If the patch_op has a chassis resident port, it means
> > > > +     *    - its peer is a distributed gateway port (DGP) and
> > > > +     *    - routing is centralized for the DGP's networks on
> > > > +     *      the configured gateway chassis.
> > > > +     *
> > > > +     * If that's the case, make sure that the packets destined to
> > > > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > > > +     * */
> > > > +
> > > > +    if (has_cr_port) {
> > > > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > > > +                      patch_op->cr_port->json_key);
> > > > +    }
> > > >
> > > >      /* Send a the packet to the router pipeline.  If the switch has
> > > non-router
> > > >       * ports then flood it there as well.
> > > > @@ -8215,6 +8312,31 @@ build_lswitch_rport_arp_req_flow(
> > > >                                  lflow_ref);
> > > >      }
> > > >
> > > > +    if (has_cr_port) {
> > > > +        ds_clear(&match);
> > > > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)",
> > > ds_cstr(&m),
> > > > +                      patch_op->cr_port->json_key);
> > > > +        ds_clear(&actions);
> > > > +        if (od->n_router_ports != od->nbs->n_ports) {
> > > > +            ds_put_format(&actions, "clone {outport = %s; output;
> }; "
> > > > +                                    "outport = \""MC_FLOOD_L2"\";
> > > output;",
> > > > +                          patch_op->cr_port->json_key);
> > > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > > +                                    priority, ds_cstr(&match),
> > > > +                                    ds_cstr(&actions), stage_hint,
> > > > +                                    lflow_ref);
> > > > +        } else {
> > > > +            ds_put_format(&actions, "outport = %s; output;",
> > > > +                          patch_op->cr_port->json_key);
> > > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > > +                                    priority, ds_cstr(&match),
> > > > +                                    ds_cstr(&actions),
> > > > +                                    stage_hint,
> > > > +                                    lflow_ref);
> > > > +        }
> > > > +    }
> > > > +
> > > > +    ds_destroy(&m);
> > > >      ds_destroy(&match);
> > > >      ds_destroy(&actions);
> > > >  }
> > > > @@ -9585,7 +9707,11 @@ build_lswitch_ip_unicast_lookup(struct
> ovn_port
> > > *op,
> > > >                                  struct ds *actions, struct ds
> *match)
> > > >  {
> > > >      ovs_assert(op->nbsp);
> > > > -    if (lsp_is_external(op->nbsp)) {
> > > > +
> > > > +    /* Note: A switch port can also have a chassis resident derived
> > > port.
> > > > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > > > +     * adding unicast lookup flows for this port. */
> > > > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> > > >          return;
> > > >      }
> > > >
> > > > @@ -9603,8 +9729,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port
> > > *op,
> > > >                             "outport = \""MC_UNKNOWN "\"; output;"
> > > >                           : "outport = %s; output;")
> > > >                           : debug_drop_action();
> > > > -    ds_clear(actions);
> > > > -    ds_put_format(actions, action, op->json_key);
> > > >
> > > >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> > > >          /* For ports connected to logical routers add flows to
> bypass
> > > the
> > > > @@ -9651,14 +9775,43 @@ build_lswitch_ip_unicast_lookup(struct
> ovn_port
> > > *op,
> > > >              if (add_chassis_resident_check) {
> > > >                  ds_put_format(match, " && is_chassis_resident(%s)",
> > > json_key);
> > > >              }
> > > > +        } else if (op->cr_port) {
> > > > +            /* If the op has a chassis resident port, it means
> > > > +             *   - its peer is a distributed gateway port (DGP) and
> > > > +             *   - routing is centralized for the DGP's networks on
> > > > +             *     the configured gateway chassis.
> > > > +             *
> > > > +             * If that's the case, make sure that the packets
> destined
> > > to
> > > > +             * the DGP's MAC are sent to the chassis where the DGP
> > > resides.
> > > > +             * */
> > > > +            ds_clear(actions);
> > > > +            ds_put_format(actions, action, op->cr_port->json_key);
> > > > +
> > > > +            struct ds m = DS_EMPTY_INITIALIZER;
> > > > +            ds_put_format(&m, "eth.dst == %s &&
> > > !is_chassis_resident(%s)",
> > > > +                          op->peer->lrp_networks.ea_s,
> > > > +                          op->cr_port->json_key);
> > > > +
> > > > +            ovn_lflow_add_with_hint(lflows, op->od,
> > > > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > > > +                                    ds_cstr(&m), ds_cstr(actions),
> > > > +                                    &op->nbsp->header_,
> > > > +                                    op->lflow_ref);
> > > > +            ds_destroy(&m);
> > > > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > > > +                          op->cr_port->json_key);
> > > >          }
> > > >
> > > > +        ds_clear(actions);
> > > > +        ds_put_format(actions, action, op->json_key);
> > > >          ovn_lflow_add_with_hint(lflows, op->od,
> > > >                                  S_SWITCH_IN_L2_LKUP, 50,
> > > >                                  ds_cstr(match), ds_cstr(actions),
> > > >                                  &op->nbsp->header_,
> > > >                                  op->lflow_ref);
> > > >      } else {
> > > > +        ds_clear(actions);
> > > > +        ds_put_format(actions, action, op->json_key);
> > > >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> > > >              ds_clear(match);
> > > >              ds_put_format(match, "eth.dst == %s",
> > > op->lsp_addrs[i].ea_s);
> > > > @@ -11772,6 +11925,14 @@ build_lrouter_port_nat_arp_nd_flow(struct
> > > ovn_port *op,
> > > >          return;
> > > >      }
> > > >
> > > > +    if (op->peer && op->peer->cr_port) {
> > > > +        /* We don't add the below flows if the router port's peer
> has
> > > > +         * a chassisresident port.  That's because routing is
> > > centralized on
> > > > +         * the gateway chassis for the router port networks/subnets.
> > > > +         */
> > > > +        return;
> > > > +    }
> > > > +
> > > >      /* Mac address to use when replying to ARP/NS. */
> > > >      const char *mac_s = REG_INPORT_ETH_ADDR;
> > > >      struct eth_addr mac;
> > > > @@ -15158,6 +15319,16 @@ lrouter_check_nat_entry(const struct
> > > ovn_datapath *od,
> > > >      /* For distributed router NAT, determine whether this NAT rule
> > > >       * satisfies the conditions for distributed NAT processing. */
> > > >      *distributed = false;
> > > > +
> > > > +    /* NAT cannnot be distributed if the DGP's peer
> > > > +     * has a chassisresident port (as the routing is centralized
> > > > +     * on the gateway chassis for the DGP's networks/subnets.)
> > > > +     */
> > >
> >
> > Hi Numan,
> >
> > I assume that this implementation as a whole will not affect the
> stateless
> > use case in which the logical_router has multiple DGPs for different
> > ovn-chassis (chassisresident port), and with a stateless NAT rule so that
> > the traffic is decentralized via ECMP, right?
>
> Hi Roberto,
>
> In order for this feature to kick in,  the option 'centralize_routing
> should be set on
> the dgw router port and the router should only have one DGP.  Your use case
> will not be affected since you'd be having multiple DGPs.
>
> Thanks
> Numan
>

Sure, thanks for clarifying this.

Best regards,
Roberto


>
> >
> > I mean, if I have a dgw_port peer for each chassis and configure a
> > stateless NAT on the router, I will continue creating the static DNAT and
> > SNAT path regardless of the port/chassis that the traffic passes through,
> > right? otherwise, this will make it impossible to balance traffic across
> > different chassis for the same logical_router.
> >
> > Kind regards,
> > Roberto
> >
> >
> > > > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > > > +    if (l3dgw_port && l3dgw_port->peer &&
> l3dgw_port->peer->cr_port) {
> > > > +        return 0;
> > > > +    }
> > > > +
> > > >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> > > >          nat->logical_port && nat->external_mac) {
> > > >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > > > diff --git a/northd/northd.h b/northd/northd.h
> > > > index d4a8d75abc..d7c9655916 100644
> > > > --- a/northd/northd.h
> > > > +++ b/northd/northd.h
> > > > @@ -325,6 +325,7 @@ struct ovn_datapath {
> > > >       * will be NULL. */
> > > >      struct ovn_port **l3dgw_ports;
> > > >      size_t n_l3dgw_ports;
> > > > +    size_t n_allocated_l3dgw_ports;
> > > >
> > > >      /* router datapath has a logical port with redirect-type set to
> > > bridged. */
> > > >      bool redirect_bridged;
> > > > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > > > index 9552534f6d..794e2fb961 100644
> > > > --- a/ovn-nb.xml
> > > > +++ b/ovn-nb.xml
> > > > @@ -3451,6 +3451,40 @@ or
> > > >            <ref column="options" key="gateway_mtu"/> option.
> > > >          </p>
> > > >        </column>
> > > > +
> > > > +      <column name="options" key="centralize_routing"
> > > > +              type='{"type": "boolean"}'>
> > > > +        <p>
> > > > +          This option is applicable only if the router port is a
> > > > +          distributed gateway port i.e if the <ref
> > > table="Logical_Router_Port"
> > > > +          column="ha_chassis_group"/> column or
> > > > +          <ref table="Logical_Router_Port"
> column="gateway_chassis"/>
> > > > +          is set.
> > > > +        </p>
> > > > +
> > > > +        <p>
> > > > +          If set to <code>true</code>, routing for the router port's
> > > > +          networks (set in the column <ref
> table="Logical_Router_Port"
> > > > +          column="networks"/>) is centralized on the gateway chassis
> > > > +          which claims this distributed gateway port.
> > > > +        </p>
> > > > +
> > > > +        <p>
> > > > +          Additionally for this option to take effect, below
> conditions
> > > > +          must be met:
> > > > +        </p>
> > > > +
> > > > +        <ul>
> > > > +          <li>
> > > > +            The Logical router has only one distributed gateway
> port.
> > > > +          </li>
> > > > +
> > > > +          <li>
> > > > +            The router port's peer logical switch has no localnet
> ports.
> > > > +          </li>
> > > > +
> > > > +        </ul>
> > > > +      </column>
> > > >      </group>
> > > >
> > > >      <group title="Attachment">
> > > > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > > > index ef41087ae3..df99f01b64 100644
> > > > --- a/tests/multinode-macros.at
> > > > +++ b/tests/multinode-macros.at
> > > > @@ -73,7 +73,7 @@ m_count_rows() {
> > > >  m_check_row_count() {
> > > >      local db=$(parse_db $1) table=$(parse_table $1); shift
> > > >      local count=$1; shift
> > > > -    local found=$(m_count_rows $c $db:$table "$@")
> > > > +    local found=$(m_count_rows $db:$table "$@")
> > > >      echo
> > > >      echo "Checking for $count rows in $db $table${1+ with $*}...
> found
> > > $found"
> > > >      if test "$count" != "$found"; then
> > > > diff --git a/tests/multinode.at b/tests/multinode.at
> > > > index 1e6eeb6610..9e01a29cc2 100644
> > > > --- a/tests/multinode.at
> > > > +++ b/tests/multinode.at
> > > > @@ -1033,4 +1033,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1],
> [sh -c
> > > 'dd bs=512 count=2 if=/dev/uran
> > > >  done
> > > >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev
> > > sw0p1 | grep -q 'mtu 942'])
> > > >
> > > > +# Reset back to geneve tunnels
> > > > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > > > +do
> > > > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > > > +done
> > > > +
> > > > +AT_CLEANUP
> > > > +
> > > > +AT_SETUP([ovn multinode NAT on a provider network with no localnet
> > > ports])
> > > > +
> > > > +# Check that ovn-fake-multinode setup is up and running
> > > > +check_fake_multinode_setup
> > > > +
> > > > +# Delete the multinode NB and OVS resources before starting the
> test.
> > > > +cleanup_multinode_resources
> > > > +
> > > > +check multinode_nbctl ls-add sw0
> > > > +check multinode_nbctl lsp-add sw0 sw0-port1
> > > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03
> > > 10.0.0.3 1000::3"
> > > > +check multinode_nbctl lsp-add sw0 sw0-port2
> > > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04
> > > 10.0.0.4 1000::4"
> > > > +
> > > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1
> > > 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2
> > > 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > > > +
> > > > +m_wait_for_ports_up
> > > > +
> > > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2
> > > 10.0.0.4 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +# Create the second logical switch with one port
> > > > +check multinode_nbctl ls-add sw1
> > > > +check multinode_nbctl lsp-add sw1 sw1-port1
> > > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03
> > > 20.0.0.3 2000::3"
> > > > +
> > > > +# Create a logical router and attach both logical switches
> > > > +check multinode_nbctl lr-add lr0
> > > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01
> 10.0.0.1/24
> > > 1000::a/64
> > > > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > > > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > > +
> > > > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02
> 20.0.0.1/24
> > > 2000::a/64
> > > > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > > > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > > > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > > > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > > +
> > > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1
> > > 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > > > +
> > > > +# create exteranl connection for N/S traffic
> > > > +check multinode_nbctl ls-add public
> > > > +check multinode_nbctl lsp-add public ln-public
> > > > +check multinode_nbctl lsp-set-type ln-public localnet
> > > > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > > > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > > > +
> > > > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01
> > > 172.20.0.100/24
> > > > +check multinode_nbctl lsp-add public public-lr0
> > > > +check multinode_nbctl lsp-set-type public-lr0 router
> > > > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > > > +check multinode_nbctl lsp-set-options public-lr0
> router-port=lr0-public
> > > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > > > +
> > > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110
> > > 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120
> 20.0.0.3
> > > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > > > +
> > > > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > > > +check multinode_nbctl lsp-add public public-port1
> > > > +check multinode_nbctl lsp-set-addresses public-port1
> "60:54:00:00:00:03
> > > 172.168.0.50"
> > > > +
> > > > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1
> > > 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > > > +
> > > > +check multinode_nbctl --wait=hv sync
> > > > +
> > > > +# First do basic ping tests before deleting the localnet port -
> > > ln-public.
> > > > +# Once the localnet port is deleted from public ls, routing for
> > > 172.20.0.0/24
> > > > +# is centralized on ovn-gw-1.
> > > > +
> > > > +# This function checks the North-South traffic.
> > > > +run_ns_traffic() {
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110],
> > > [ignore], [ignore])
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120],
> > > [ignore], [ignore])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.100 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.110 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.120 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.50 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.50 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120,
> > > 10.0.0.3 and 20.0.0.3
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.100 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.110 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 172.20.0.120 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 10.0.0.3 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w
> 2
> > > 20.0.0.3 | FORMAT_PING], \
> > > > +[0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +}
> > > > +
> > > > +# Test out the N-S traffic.
> > > > +run_ns_traffic
> > > > +
> > > > +# Delete the localnet port by changing the type of ln-public to VIF
> > > port.
> > > > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > > > +
> > > > +# cr-port should not be created for public-lr0 since the option
> > > > +# centralize_routing=true is not yet set for lr0-public.
> > > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > > +
> > > > +# Set the option - centralize_routing now.
> > > > +check multinode_nbctl --wait=hv set logical_router_port lr0-public
> > > options:centralize_routing=true
> > > > +
> > > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > > +m_check_column chassisredirect Port_Binding type
> > > logical_port=cr-public-lr0
> > > > +
> > > > +# Test out the N-S traffic.
> > > > +run_ns_traffic
> > > > +
> > > > +# Re-add the localnet port
> > > > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > > > +
> > > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > > +
> > > > +# Test out the N-S traffic.
> > > > +run_ns_traffic
> > > > +
> > > > +# Delete the ln-public port this time.
> > > > +check multinode_nbctl --wait=hv lsp-del ln-public
> > > > +
> > > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > > +m_check_column chassisredirect Port_Binding type
> > > logical_port=cr-public-lr0
> > > > +
> > > > +# Test out the N-S traffic.
> > > > +run_ns_traffic
> > > > +
> > > >  AT_CLEANUP
> > > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > > > index a389d19886..5445fff494 100644
> > > > --- a/tests/ovn-northd.at
> > > > +++ b/tests/ovn-northd.at
> > > > @@ -2188,7 +2188,7 @@ match=(inport == "lrp-public" && arp.op == 1 &&
> > > arp.tpa == 43.43.43.4 && is_chas
> > > >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2;
> /*
> > > ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa
> <->
> > > arp.spa; outport = inport; flags.loopback = 1; output;)
> > > >  ])
> > > >
> > > > -# xreg0[0..47] isn't used anywhere else.
> > > > +# xreg0[[0..47]] isn't used anywhere else.
> > > >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE
> > > 'lr_in_admission|lr_in_ip_input'], [1], [])
> > > >
> > > >  AT_CLEANUP
> > > > @@ -5524,13 +5524,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows |
> grep
> > > "192.168.4.100" | grep "_MC_flo
> > > >
> > > >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> > > >
> > > > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1
> > > localnet
> > > >
> > > >  ovn-sbctl lflow-list ls1 > ls1_lflows
> > > >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0],
> [dnl
> > > >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > > action=(outport = get_fdb(eth.dst); next;)
> > > >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > > action=(handle_svc_check(inport);)
> > > > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")),
> action=(outport =
> > > "ls1-ro1"; output;)
> > > >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> > > >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > > action=(outport = "_MC_flood"; output;)
> > > >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)),
> > > action=(outport = "_MC_flood_l2"; output;)
> > > > @@ -12721,3 +12722,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep
> > > lr_in_dnat | ovn_strip_lflows], [0], [d
> > > >
> > > >  AT_CLEANUP
> > > >  ])
> > > > +
> > > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > > +AT_SETUP([NAT on a provider network with no localnet ports])
> > > > +AT_KEYWORDS([NAT])
> > > > +ovn_start
> > > > +
> > > > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > > > +check ovn-nbctl lsp-add sw0 sw0-port1
> > > > +check ovn-nbctl lr-add lr0
> > > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > > > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > > > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > > > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > > +
> > > > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > > > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > > > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > > > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > > > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > > +
> > > > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > > > +check ovn-nbctl ls-add public
> > > > +check ovn-nbctl lsp-add public pub-p1
> > > > +
> > > > +# localnet port
> > > > +check ovn-nbctl lsp-add public ln-public
> > > > +check ovn-nbctl lsp-set-type ln-public localnet
> > > > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > > > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > > > +
> > > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02
> > > 172.168.0.10/24
> > > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > > > +
> > > > +check ovn-nbctl lsp-add public public-lr0
> > > > +check ovn-nbctl lsp-set-type public-lr0 router
> > > > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > > +
> > > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3
> > > sw0-port1 30:54:00:00:00:03
> > > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > > > +
> > > > +check ovn-nbctl --wait=sb sync
> > > > +
> > > > +check_flows_no_cr_port_for_public_lr0() {
> > > > +  # check that there is no port binding cr-public-lr0
> > > > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > > +
> > > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > > +  ovn-sbctl dump-flows public > publicflows
> > > > +
> > > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> > > action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=100  ,
> match=(vlan.present ||
> > > eth.src[[40]]), action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> > > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > > icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> > > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > > icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> > > !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> > > action=(outport <-> inport; inport = "lr0-public"; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:01; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:02 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:03; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02;
> next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01;
> next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03;
> next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  ,
> match=(ip4.src_mcast
> > > ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> > > 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> > > "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl ==
> {0,
> > > 1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag),
> action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254;
> outport =
> > > "lr0-public"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> > > ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> > > ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl ==
> {0,
> > > 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {10.0.0.1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {172.168.0.10}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {20.0.0.1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> > > ip6.mcast), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=83   ,
> > > match=(ip6.mcast_rsvd), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> > > nd_ra), action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> > > 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")),
> action=(eth.dst
> > > = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */
> arp.tha =
> > > arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport =
> inport;
> > > flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02}
> &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > > is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> > > xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> > > inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa ==
> 10.0.0.0/24),
> > > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <->
> arp.spa;
> > > outport = inport; flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> > > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > > outport = inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa ==
> 20.0.0.0/24),
> > > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <->
> arp.spa;
> > > outport = inport; flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> > > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > > outport = inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst
> <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src;
> eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
> arp.sha =
> > > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport;
> flags.loopback =
> > > 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src;
> eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.100 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> > > action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"),
> action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> > > action=(get_arp(outport, reg0); next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> > > action=(get_nd(outport, xxreg0); next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 30:54:00:00:00:03; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> > > ip6.mcast), action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > > 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")),
> > > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> > > "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > > 30:54:00:00:00:03; ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> > > action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> > > action=(next;)
> > > > +  table=??(lr_out_snat        ), priority=153  , match=(ip &&
> ip4.src
> > > == 10.0.0.0/24 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.100);)
> > > > +  table=??(lr_out_snat        ), priority=153  , match=(ip &&
> ip4.src
> > > == 20.0.0.0/24 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.100);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")
> > > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > > ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.100 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> > > [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > > action=(outport = get_fdb(eth.dst); next;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > > action=(handle_svc_check(inport);)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")),
> action=(outport
> > > = "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > > action=(outport = "_MC_flood"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone
> {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 30:54:00:00:00:03; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"),
> action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > > 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")),
> > > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
> arp.sha =
> > > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport;
> flags.loopback =
> > > 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src;
> eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")
> > > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > > ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > > 30:54:00:00:00:03; ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +}
> > > > +
> > > > +check_flows_cr_port_for_public_lr0() {
> > > > +  # check that there is port binding cr-public-lr0
> > > > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > > +  check_column chassisredirect Port_Binding type
> > > logical_port=cr-public-lr0
> > > > +
> > > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > > +  ovn-sbctl dump-flows public > publicflows
> > > > +
> > > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_admission    ), priority=0    , match=(1),
> > > action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=100  ,
> match=(vlan.present ||
> > > eth.src[[40]]), action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 &&
> > > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > > icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 &&
> > > icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 &&
> > > icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 &&
> > > !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1),
> > > action=(outport <-> inport; inport = "lr0-public"; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:01; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:02 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:03; next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02;
> next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01;
> next;)
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast &&
> > > inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03;
> next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src ==
> > > {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  ,
> match=(ip4.src_mcast
> > > ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst ==
> > > 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546),
> action=(reg0 =
> > > 0; handle_dhcpv6_reply;)
> > > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport ==
> > > "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl ==
> {0,
> > > 1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag),
> action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254;
> outport =
> > > "lr0-public"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ;
> > > ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport ==
> > > "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4
> > > {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code
> = 0;
> > > /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ;
> > > ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl ==
> {0,
> > > 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {10.0.0.1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {172.168.0.10}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst ==
> > > {20.0.0.1}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst ==
> > > {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast ||
> > > ip6.mcast), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=83   ,
> > > match=(ip6.mcast_rsvd), action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs ||
> > > nd_ra), action=(next;)
> > > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd),
> > > action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa ==
> > > 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]];
> > > arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha =
> xreg0[[0..47]];
> > > arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02}
> &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > > is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src =
> > > xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport =
> > > inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa ==
> 10.0.0.0/24),
> > > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <->
> arp.spa;
> > > outport = inport; flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router {
> > > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > > outport = inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa ==
> 20.0.0.0/24),
> > > action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP
> > > reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <->
> arp.spa;
> > > outport = inport; flags.loopback = 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport ==
> > > "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} &&
> > > nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router {
> > > eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]];
> > > outport = inport; flags.loopback = 1; output; };)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst
> <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst ==
> > > 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <->
> > > ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst ==
> > > fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0),
> > > action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129;
> flags.loopback
> > > = 1; next; )
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.100 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_in_defrag       ), priority=0    , match=(1),
> > > action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_dnat         ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> > > action=(get_arp(outport, reg0); next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> > > action=(get_nd(outport, xxreg0); next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast ||
> > > ip6.mcast), action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport ==
> > > "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_out_undnat      ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows],
> [0],
> > > [dnl
> > > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1),
> > > action=(next;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_out_snat        ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns),
> > > action=(next;)
> > > > +  table=??(lr_out_snat        ), priority=153  , match=(ip &&
> ip4.src
> > > == 10.0.0.0/24 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.100);)
> > > > +  table=??(lr_out_snat        ), priority=153  , match=(ip &&
> ip4.src
> > > == 20.0.0.0/24 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.100);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0],
> [dnl
> > > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1),
> > > action=(next;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.100 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0],
> > > [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1),
> > > action=(outport = get_fdb(eth.dst); next;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst ==
> > > $svc_monitor_mac && (tcp || icmp || icmp6)),
> > > action=(handle_svc_check(inport);)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")),
> > > action=(outport = "cr-public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")),
> action=(outport
> > > = "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast),
> > > action=(outport = "_MC_flood"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.10 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.100 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && nd_ns && nd.target == fe80::200:ff:fe00:ff02 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > !is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-public-lr0")), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +}
> > > > +
> > > > +# Check that the lflows are as expected when public has localnet
> port.
> > > > +check_flows_no_cr_port_for_public_lr0
> > > > +
> > > > +# Remove the localnet port from public logical switch.
> > > > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > > > +
> > > > +# Check that the lflows are as expected and there is no cr port
> > > > +# created for "public-lr0"  when public has no localnet port
> > > > +# since public doesn't have the option
> "overlay_provider_network=true"
> > > > +# set.
> > > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > > +
> > > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > > +ovn-sbctl dump-flows public > publicflows
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 30:54:00:00:00:03; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"),
> action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > > 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")),
> > > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
> arp.sha =
> > > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport;
> flags.loopback =
> > > 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src;
> eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")
> > > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > > ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > > 30:54:00:00:00:03; ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +
> > > > +
> > > > +# Set the option "centralize_routing=true" for lr0-public.
> > > > +check ovn-nbctl --wait=sb set logical_router_port lr0-public
> > > options:centralize_routing=true
> > > > +
> > > > +# Check that the lflows are as expected and there is cr port created
> > > for public-lr0.
> > > > +check_flows_cr_port_for_public_lr0
> > > > +
> > > > +# Set the type of ln-public back to localnet
> > > > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > > > +
> > > > +# Check that the lflows are as expected when public has localnet
> port.
> > > > +check_flows_no_cr_port_for_public_lr0
> > > > +
> > > > +# Delete the localnet port
> > > > +check ovn-nbctl --wait=sb lsp-del ln-public
> > > > +
> > > > +# Check that the lflows are as expected when public has no localnet
> > > port.
> > > > +check_flows_cr_port_for_public_lr0
> > > > +
> > > > +# Create multiple gateway ports.  chassisresident port should not be
> > > > +# created for 'public-lr0' even if there is no localnet port on
> 'public'
> > > > +# logical switch.
> > > > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > > > +# check that there is no port binding cr-public-lr0
> > > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > > +
> > > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > > +ovn-sbctl dump-flows public > publicflows
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && inport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst =
> > > 30:54:00:00:00:03; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport ==
> > > "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst =
> > > 00:00:00:00:ff:02; next;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110),
> > > action=(drop;)
> > > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport ==
> > > "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120),
> > > action=(drop;)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"),
> action=(ct_dnat(10.0.0.3);)
> > > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src ==
> > > 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")),
> > > action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1
> &&
> > > arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120),
> action=(drop;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 &&
> > > is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src =
> > > 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
> arp.sha =
> > > 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport;
> flags.loopback =
> > > 1; output;)
> > > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport ==
> > > "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 &&
> > > is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src;
> eth.src =
> > > xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha
> =
> > > xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback
> = 1;
> > > output;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip &&
> ip4.dst
> > > == 172.168.0.120 && inport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.110 && outport == "lr0-public" &&
> > > is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst ==
> > > 172.168.0.120 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear;
> inport =
> > > outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback
> = 1;
> > > reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0;
> reg7
> > > = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress,
> table=??);
> > > };)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public" &&
> is_chassis_resident("sw0-port1")
> > > && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03;
> > > ct_snat(172.168.0.110);)
> > > > +  table=??(lr_out_snat        ), priority=161  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)),
> > > action=(ct_snat(172.168.0.120);)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 10.0.0.3 && outport == "lr0-public"), action=(eth.src =
> > > 30:54:00:00:00:03; ct_dnat;)
> > > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip &&
> ip4.src
> > > == 20.0.0.3 && outport == "lr0-public" &&
> > > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > > +])
> > > > +
> > > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3"
> -e
> > > "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows |
> > > ovn_strip_lflows], [0], [dnl
> > > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> > > 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")),
> action=(outport =
> > > "public-lr0"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src ==
> > > {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3
> ||
> > > nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]]
> == 0
> > > && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport =
> > > "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > > +])
> > > > +
> > > > +AT_CLEANUP
> > > > +])
> > > > diff --git a/tests/ovn.at b/tests/ovn.at
> > > > index 185ba4a21e..21d7484f9e 100644
> > > > --- a/tests/ovn.at
> > > > +++ b/tests/ovn.at
> > > > @@ -21232,10 +21232,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set
> > > Logical_Switch_Port rp-sw0 \
> > > >      type=router options:router-port=sw0 \
> > > >      -- lsp-set-addresses rp-sw0 router
> > > >
> > > > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24
> > > 2002:0:0:0:0:0:0:1/64 \
> > > > -    -- lrp-set-gateway-chassis sw1 hv2
> > > > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24
> > > 2002:0:0:0:0:0:0:1/64 \
> > > > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> > > >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > > > -    type=router options:router-port=sw1 \
> > > > +    type=router options:router-port=lr0-sw1 \
> > > >      -- lsp-set-addresses rp-sw1 router
> > > >
> > > >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > > > @@ -21247,6 +21247,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> > > >  ovn-nbctl lsp-add sw1 sw1-p0 \
> > > >      -- lsp-set-addresses sw1-p0 unknown
> > > >
> > > > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > > > +
> > > >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> > > >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> > > >
> > > > --
> > > > 2.45.2
> > > >
> > > > _______________________________________________
> > > > dev mailing list
> > > > dev@openvswitch.org
> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > >
> >
> > --
> >
> >
> >
> >
> > _‘Esta mensagem é direcionada apenas para os endereços constantes no
> > cabeçalho inicial. Se você não está listado nos endereços constantes no
> > cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa
> > mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas
> estão
> > imediatamente anuladas e proibidas’._
> >
> >
> > * **‘Apesar do Magazine Luiza tomar
> > todas as precauções razoáveis para assegurar que nenhum vírus esteja
> > presente nesse e-mail, a empresa não poderá aceitar a responsabilidade
> por
> > quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*
> >
> >
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Mark Michelson July 25, 2024, 8:30 p.m. UTC | #5
Aside from your note about the wrong NEWS item, this looks good to me.

Acked-by: Mark Michelson <mmichels@redhat.com>

On 7/9/24 19:25, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> Consider a deployment with the below logical resources:
> 
> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>     port ln-public.
> 2. A logical router 'R'
> 3. Logical switch 'public' connected to R via logical switch/router port
>     peers (public-R and R-public).
> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> 5. NATs (dnat_and_snat) configured in 'R'.
> 6. And a few overlay logical switches S1, S2 to R.
> 
> Any traffic from logical port - P1 of public logical switch destined to
> S1 or S2's logical ports goes out of the source chassis
> (where P1 resides) via the localnet port and reaches the gateway chassis
> which handles the routing.
> 
> There are couple of traffic flow scenarios which doesn't work if the
> logical switch 'public' doesn't have a localnet port.
> 
> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>     dropped in the source chassis.  The packet enters the router R's
>     pipeline, but it gets dropped in the 'lr_in_admission' stage since
>     the logical flow to allow traffic destined to the distributed gateway
>     port MAC is installed only on the gateway chassis.
> 
> 2. NAT doesn't work as expected.
> 
> In order to suppose this use case (of a logical switch not having a
> localnet port, but has a distributed gateway port and NATs), this patch
> supports the option 'centralize_routing', which can be configured on
> the distributed gateway port (R-public in the example above).
> If this option is set, then routing is centralized on the gateway
> chassis for the traffic destined to the R-public's networks
> (172.16.0.0/24 for the above example).  Traffic from P1 will be
> tunnelled to the gateway chassis.
> 
> ovn-northd creates a chassisresident port (cr-public-R) for the
> logical switch port - public-R, along with cr-R-public inorder to
> centralize the traffic.
> 
> This feature gets enabled for the distributed gateway port R-public if
>    - The above option is set to true in the R-public's options column.
>    - The logical switch 'public' doesn't have any localnet ports.
>    - And R-public is the only distributed gateway port of R.
> 
> Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic
> and centralized on the gateway chassis.
> 
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>   NEWS                      |   2 +
>   controller/physical.c     |   4 +
>   northd/northd.c           | 257 +++++++++++++++----
>   northd/northd.h           |   1 +
>   ovn-nb.xml                |  34 +++
>   tests/multinode-macros.at |   2 +-
>   tests/multinode.at        | 177 +++++++++++++
>   tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>   tests/ovn.at              |   8 +-
>   9 files changed, 951 insertions(+), 50 deletions(-)
> 
> diff --git a/NEWS b/NEWS
> index 3e392ff08b..472445a188 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -38,6 +38,8 @@ Post v24.03.0
>       ability to disable "VXLAN mode" to extend available tunnel IDs space for
>       datapaths from 4095 to 16711680.  For more details see man ovn-nb(5) for
>       mentioned option.
> +  - Added Overlay provider network support to a logical switch if
> +    the config "overlay_provider_network" is set to true.
>   
>   OVN v24.03.0 - 01 Mar 2024
>   --------------------------
> diff --git a/controller/physical.c b/controller/physical.c
> index 22756810fd..e3a316989a 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1608,6 +1608,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                       ct_zones);
>               put_zones_ofpacts(&zone_ids, ofpacts_p);
>   
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>               /* Resubmit to table 41. */
>               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>           }
> diff --git a/northd/northd.c b/northd/northd.c
> index 6898daa00d..9b52d5a3c0 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2099,6 +2099,55 @@ parse_lsp_addrs(struct ovn_port *op)
>       }
>   }
>   
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports,
> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> +        ovn_port_set_nb(crp, NULL, op->nbrp);
> +        ovs_list_remove(&crp->list);
> +        ovs_list_push_back(both_dbs, &crp->list);
> +    } else {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +        ovs_list_push_back(nb_only, &crp->list);
> +    }
> +
> +    crp->primary_port = op;
> +    op->cr_port = crp;
> +    crp->od = op->od;
> +    free(redirect_name);
> +
> +    return crp;
> +}
> +
> +/* Returns true if chassis resident port needs to be created for
> + * op's peer logical switch.  False otherwise.
> + *
> + * Chassis resident port needs to be created if the following
> + * conditionsd are met:
> + *   - op is a distributed gateway port
> + *   - op has the option 'centralize_routing' set to true
> + *   - op is the only distributed gateway port attached to its
> + *     router
> + *   - op's peer logical switch has no localnet ports.
> + */
> +static bool
> +peer_needs_cr_port_creation(struct ovn_port *op)
> +{
> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +        && !op->peer->od->n_localnet_ports) {
> +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> +    }
> +
> +    return false;
> +}
> +
>   static void
>   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2206,9 +2255,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>           }
>       }
> +
> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>           ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>           for (size_t i = 0; i < od->nbr->n_ports; i++) {
>               const struct nbrec_logical_router_port *nbrp
>                   = od->nbr->ports[i];
> @@ -2272,10 +2322,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       redirect_type && !strcasecmp(redirect_type, "bridged");
>               }
>   
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                   const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                  "chassis");
>                   if (gw_chassis) {
> @@ -2284,34 +2331,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                    "gateway port configured on port %s "
>                                    "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                   } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->primary_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&dgps, op);
>                   }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>              }
>           }
>       }
> @@ -2368,12 +2390,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                           arp_proxy, op->nbsp->name);
>                   }
>               }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>           } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>               if (peer) {
> @@ -2394,6 +2410,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>           }
>       }
>   
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> +        ovs_assert(crp);
> +
> +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                        &od->n_allocated_l3dgw_ports,
> +                                        sizeof *od->l3dgw_ports);
> +        }
> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the distributed gateway port's (DGP)
> +     * peer if
> +     *  - DGP's router has only one DGP and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports and
> +     *  - option 'centralize_routing' is set to true for the DGP.
> +     *
> +     * This is required to support
> +     *   - NAT via geneve (for the overlay provider networks) and
> +     *   - to centralize routing on the gateway chassis for the traffic
> +     *     destined to the DGP's networks.
> +     *
> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> +     * of a logical router.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        if (peer_needs_cr_port_creation(op)) {
> +            create_cr_port(op->peer, ports, both, nb_only);
> +        }
> +    }
> +    hmapx_destroy(&dgps);
> +
>       /* Wait until all ports have been connected to add to IPAM since
>        * it relies on proper peers to be set
>        */
> @@ -3176,16 +3243,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>                * type "l3gateway". */
>               if (chassis) {
>                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->primary_port->peer);
> +                ovs_assert(op->primary_port->peer->cr_port);
> +                ovs_assert(op->primary_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> +
>               } else {
>                   sbrec_port_binding_set_type(op->sb, "patch");
>               }
>   
>               const char *router_port = smap_get(&op->nbsp->options,
>                                                  "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                   struct smap new;
>                   smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                       smap_add(&new, "peer", router_port);
>                   }
>                   if (chassis) {
> @@ -8191,9 +8270,27 @@ build_lswitch_rport_arp_req_flow(
>       struct lflow_ref *lflow_ref)
>   {
>       struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>       struct ds actions = DS_EMPTY_INITIALIZER;
>   
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    /* If the patch_op has a chassis resident port, it means
> +     *    - its peer is a distributed gateway port (DGP) and
> +     *    - routing is centralized for the DGP's networks on
> +     *      the configured gateway chassis.
> +     *
> +     * If that's the case, make sure that the packets destined to
> +     * the DGP's MAC are sent to the chassis where the DGP resides.
> +     * */
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>   
>       /* Send a the packet to the router pipeline.  If the switch has non-router
>        * ports then flood it there as well.
> @@ -8215,6 +8312,31 @@ build_lswitch_rport_arp_req_flow(
>                                   lflow_ref);
>       }
>   
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>       ds_destroy(&match);
>       ds_destroy(&actions);
>   }
> @@ -9585,7 +9707,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                   struct ds *actions, struct ds *match)
>   {
>       ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +
> +    /* Note: A switch port can also have a chassis resident derived port.
> +     * Check if 'op' is a chassis resident dervied port. If so, skip
> +     * adding unicast lookup flows for this port. */
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>           return;
>       }
>   
> @@ -9603,8 +9729,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                              "outport = \""MC_UNKNOWN "\"; output;"
>                            : "outport = %s; output;")
>                            : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>   
>       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>           /* For ports connected to logical routers add flows to bypass the
> @@ -9651,14 +9775,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>               if (add_chassis_resident_check) {
>                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>               }
> +        } else if (op->cr_port) {
> +            /* If the op has a chassis resident port, it means
> +             *   - its peer is a distributed gateway port (DGP) and
> +             *   - routing is centralized for the DGP's networks on
> +             *     the configured gateway chassis.
> +             *
> +             * If that's the case, make sure that the packets destined to
> +             * the DGP's MAC are sent to the chassis where the DGP resides.
> +             * */
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>           }
>   
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           ovn_lflow_add_with_hint(lflows, op->od,
>                                   S_SWITCH_IN_L2_LKUP, 50,
>                                   ds_cstr(match), ds_cstr(actions),
>                                   &op->nbsp->header_,
>                                   op->lflow_ref);
>       } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>               ds_clear(match);
>               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11772,6 +11925,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>           return;
>       }
>   
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the router port networks/subnets.
> +         */
> +        return;
> +    }
> +
>       /* Mac address to use when replying to ARP/NS. */
>       const char *mac_s = REG_INPORT_ETH_ADDR;
>       struct eth_addr mac;
> @@ -15158,6 +15319,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>       /* For distributed router NAT, determine whether this NAT rule
>        * satisfies the conditions for distributed NAT processing. */
>       *distributed = false;
> +
> +    /* NAT cannnot be distributed if the DGP's peer
> +     * has a chassisresident port (as the routing is centralized
> +     * on the gateway chassis for the DGP's networks/subnets.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>           nat->logical_port && nat->external_mac) {
>           if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index d4a8d75abc..d7c9655916 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>        * will be NULL. */
>       struct ovn_port **l3dgw_ports;
>       size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>   
>       /* router datapath has a logical port with redirect-type set to bridged. */
>       bool redirect_bridged;
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index 9552534f6d..794e2fb961 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -3451,6 +3451,40 @@ or
>             <ref column="options" key="gateway_mtu"/> option.
>           </p>
>         </column>
> +
> +      <column name="options" key="centralize_routing"
> +              type='{"type": "boolean"}'>
> +        <p>
> +          This option is applicable only if the router port is a
> +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> +          column="ha_chassis_group"/> column or
> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> +          is set.
> +        </p>
> +
> +        <p>
> +          If set to <code>true</code>, routing for the router port's
> +          networks (set in the column <ref table="Logical_Router_Port"
> +          column="networks"/>) is centralized on the gateway chassis
> +          which claims this distributed gateway port.
> +        </p>
> +
> +        <p>
> +          Additionally for this option to take effect, below conditions
> +          must be met:
> +        </p>
> +
> +        <ul>
> +          <li>
> +            The Logical router has only one distributed gateway port.
> +          </li>
> +
> +          <li>
> +            The router port's peer logical switch has no localnet ports.
> +          </li>
> +
> +        </ul>
> +      </column>
>       </group>
>   
>       <group title="Attachment">
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index ef41087ae3..df99f01b64 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -73,7 +73,7 @@ m_count_rows() {
>   m_check_row_count() {
>       local db=$(parse_db $1) table=$(parse_table $1); shift
>       local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>       echo
>       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>       if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index 1e6eeb6610..9e01a29cc2 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1033,4 +1033,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
>   done
>   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
>   
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# centralize_routing=true is not yet set for lr0-public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option - centralize_routing now.
> +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>   AT_CLEANUP
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index a389d19886..5445fff494 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2188,7 +2188,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   AT_CLEANUP
> @@ -5524,13 +5524,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>   
>   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>   
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>   
>   ovn-sbctl lflow-list ls1 > ls1_lflows
>   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12721,3 +12722,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
>   
>   AT_CLEANUP
>   ])
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([NAT])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "centralize_routing=true" for lr0-public.
> +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 185ba4a21e..21d7484f9e 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21232,10 +21232,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>       type=router options:router-port=sw0 \
>       -- lsp-set-addresses rp-sw0 router
>   
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>       -- lsp-set-addresses rp-sw1 router
>   
>   ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21247,6 +21247,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>   ovn-nbctl lsp-add sw1 sw1-p0 \
>       -- lsp-set-addresses sw1-p0 unknown
>   
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>
diff mbox series

Patch

diff --git a/NEWS b/NEWS
index 3e392ff08b..472445a188 100644
--- a/NEWS
+++ b/NEWS
@@ -38,6 +38,8 @@  Post v24.03.0
     ability to disable "VXLAN mode" to extend available tunnel IDs space for
     datapaths from 4095 to 16711680.  For more details see man ovn-nb(5) for
     mentioned option.
+  - Added Overlay provider network support to a logical switch if
+    the config "overlay_provider_network" is set to true.
 
 OVN v24.03.0 - 01 Mar 2024
 --------------------------
diff --git a/controller/physical.c b/controller/physical.c
index 22756810fd..e3a316989a 100644
--- a/controller/physical.c
+++ b/controller/physical.c
@@ -1608,6 +1608,10 @@  consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
                                                     ct_zones);
             put_zones_ofpacts(&zone_ids, ofpacts_p);
 
+            /* Clear the MFF_INPORT.  Its possible that the same packet may
+             * go out from the same tunnel inport. */
+            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
+
             /* Resubmit to table 41. */
             put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
         }
diff --git a/northd/northd.c b/northd/northd.c
index 6898daa00d..9b52d5a3c0 100644
--- a/northd/northd.c
+++ b/northd/northd.c
@@ -2099,6 +2099,55 @@  parse_lsp_addrs(struct ovn_port *op)
     }
 }
 
+static struct ovn_port *
+create_cr_port(struct ovn_port *op, struct hmap *ports,
+               struct ovs_list *both_dbs, struct ovs_list *nb_only)
+{
+    char *redirect_name = ovn_chassis_redirect_name(
+        op->nbsp ? op->nbsp->name : op->nbrp->name);
+
+    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
+    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
+        ovn_port_set_nb(crp, NULL, op->nbrp);
+        ovs_list_remove(&crp->list);
+        ovs_list_push_back(both_dbs, &crp->list);
+    } else {
+        crp = ovn_port_create(ports, redirect_name,
+                              op->nbsp, op->nbrp, NULL);
+        ovs_list_push_back(nb_only, &crp->list);
+    }
+
+    crp->primary_port = op;
+    op->cr_port = crp;
+    crp->od = op->od;
+    free(redirect_name);
+
+    return crp;
+}
+
+/* Returns true if chassis resident port needs to be created for
+ * op's peer logical switch.  False otherwise.
+ *
+ * Chassis resident port needs to be created if the following
+ * conditionsd are met:
+ *   - op is a distributed gateway port
+ *   - op has the option 'centralize_routing' set to true
+ *   - op is the only distributed gateway port attached to its
+ *     router
+ *   - op's peer logical switch has no localnet ports.
+ */
+static bool
+peer_needs_cr_port_creation(struct ovn_port *op)
+{
+    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
+        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
+        && !op->peer->od->n_localnet_ports) {
+        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
+    }
+
+    return false;
+}
+
 static void
 join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                    struct hmap *ls_datapaths, struct hmap *lr_datapaths,
@@ -2206,9 +2255,10 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
             tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
         }
     }
+
+    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
     HMAP_FOR_EACH (od, key_node, lr_datapaths) {
         ovs_assert(od->nbr);
-        size_t n_allocated_l3dgw_ports = 0;
         for (size_t i = 0; i < od->nbr->n_ports; i++) {
             const struct nbrec_logical_router_port *nbrp
                 = od->nbr->ports[i];
@@ -2272,10 +2322,7 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                     redirect_type && !strcasecmp(redirect_type, "bridged");
             }
 
-            if (op->nbrp->ha_chassis_group ||
-                op->nbrp->n_gateway_chassis) {
-                /* Additional "derived" ovn_port crp represents the
-                 * instance of op on the gateway chassis. */
+            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
                 const char *gw_chassis = smap_get(&op->od->nbr->options,
                                                "chassis");
                 if (gw_chassis) {
@@ -2284,34 +2331,9 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                     VLOG_WARN_RL(&rl, "Bad configuration: distributed "
                                  "gateway port configured on port %s "
                                  "on L3 gateway router", nbrp->name);
-                    continue;
-                }
-
-                char *redirect_name =
-                    ovn_chassis_redirect_name(nbrp->name);
-                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
-                if (crp && crp->sb && crp->sb->datapath == od->sb) {
-                    ovn_port_set_nb(crp, NULL, nbrp);
-                    ovs_list_remove(&crp->list);
-                    ovs_list_push_back(both, &crp->list);
                 } else {
-                    crp = ovn_port_create(ports, redirect_name,
-                                          NULL, nbrp, NULL);
-                    ovs_list_push_back(nb_only, &crp->list);
-                }
-                crp->primary_port = op;
-                op->cr_port = crp;
-                crp->od = od;
-                free(redirect_name);
-
-                /* Add to l3dgw_ports in od, for later use during flow
-                 * creation. */
-                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
-                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
-                                                 &n_allocated_l3dgw_ports,
-                                                 sizeof *od->l3dgw_ports);
+                    hmapx_add(&dgps, op);
                 }
-                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
            }
         }
     }
@@ -2368,12 +2390,6 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                         arp_proxy, op->nbsp->name);
                 }
             }
-
-            /* Only used for the router type LSP whose peer is l3dgw_port */
-            if (op->peer && is_l3dgw_port(op->peer)) {
-                op->enable_router_port_acl = smap_get_bool(
-                    &op->nbsp->options, "enable_router_port_acl", false);
-            }
         } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
             struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
             if (peer) {
@@ -2394,6 +2410,57 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
         }
     }
 
+    struct hmapx_node *hmapx_node;
+    HMAPX_FOR_EACH (hmapx_node, &dgps) {
+        op = hmapx_node->data;
+        od = op->od;
+        ovs_assert(op->nbrp);
+        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
+
+        /* Additional "derived" ovn_port crp represents the instance of op on
+         * the gateway chassis. */
+        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
+        ovs_assert(crp);
+
+        /* Add to l3dgw_ports in od, for later use during flow creation. */
+        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
+            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
+                                        &od->n_allocated_l3dgw_ports,
+                                        sizeof *od->l3dgw_ports);
+        }
+        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
+
+        if (op->peer && op->peer->nbsp) {
+            /* Only used for the router type LSP whose peer is l3dgw_port */
+            op->peer->enable_router_port_acl = smap_get_bool(
+                    &op->peer->nbsp->options, "enable_router_port_acl", false);
+        }
+    }
+
+
+    /* Create chassisresident port for the distributed gateway port's (DGP)
+     * peer if
+     *  - DGP's router has only one DGP and
+     *  - Its peer is a logical switch port and
+     *  - It's peer's logical switch has no localnet ports and
+     *  - option 'centralize_routing' is set to true for the DGP.
+     *
+     * This is required to support
+     *   - NAT via geneve (for the overlay provider networks) and
+     *   - to centralize routing on the gateway chassis for the traffic
+     *     destined to the DGP's networks.
+     *
+     * Future enhancement: Support 'centralizerouting' for all the DGP's
+     * of a logical router.
+     * */
+    HMAPX_FOR_EACH (hmapx_node, &dgps) {
+        op = hmapx_node->data;
+        if (peer_needs_cr_port_creation(op)) {
+            create_cr_port(op->peer, ports, both, nb_only);
+        }
+    }
+    hmapx_destroy(&dgps);
+
     /* Wait until all ports have been connected to add to IPAM since
      * it relies on proper peers to be set
      */
@@ -3176,16 +3243,28 @@  ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
              * type "l3gateway". */
             if (chassis) {
                 sbrec_port_binding_set_type(op->sb, "l3gateway");
+            } else if (is_cr_port(op)) {
+                sbrec_port_binding_set_type(op->sb, "chassisredirect");
+                ovs_assert(op->primary_port->peer);
+                ovs_assert(op->primary_port->peer->cr_port);
+                ovs_assert(op->primary_port->peer->cr_port->sb);
+                sbrec_port_binding_set_ha_chassis_group(
+                    op->sb,
+                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
+
             } else {
                 sbrec_port_binding_set_type(op->sb, "patch");
             }
 
             const char *router_port = smap_get(&op->nbsp->options,
                                                "router-port");
-            if (router_port || chassis) {
+            if (router_port || chassis || is_cr_port(op)) {
                 struct smap new;
                 smap_init(&new);
-                if (router_port) {
+
+                if (is_cr_port(op)) {
+                    smap_add(&new, "distributed-port", op->nbsp->name);
+                } else if (router_port) {
                     smap_add(&new, "peer", router_port);
                 }
                 if (chassis) {
@@ -8191,9 +8270,27 @@  build_lswitch_rport_arp_req_flow(
     struct lflow_ref *lflow_ref)
 {
     struct ds match   = DS_EMPTY_INITIALIZER;
+    struct ds m       = DS_EMPTY_INITIALIZER;
     struct ds actions = DS_EMPTY_INITIALIZER;
 
-    arp_nd_ns_match(ips, addr_family, &match);
+    arp_nd_ns_match(ips, addr_family, &m);
+    ds_clone(&match, &m);
+
+    bool has_cr_port = patch_op->cr_port;
+
+    /* If the patch_op has a chassis resident port, it means
+     *    - its peer is a distributed gateway port (DGP) and
+     *    - routing is centralized for the DGP's networks on
+     *      the configured gateway chassis.
+     *
+     * If that's the case, make sure that the packets destined to
+     * the DGP's MAC are sent to the chassis where the DGP resides.
+     * */
+
+    if (has_cr_port) {
+        ds_put_format(&match, " && is_chassis_resident(%s)",
+                      patch_op->cr_port->json_key);
+    }
 
     /* Send a the packet to the router pipeline.  If the switch has non-router
      * ports then flood it there as well.
@@ -8215,6 +8312,31 @@  build_lswitch_rport_arp_req_flow(
                                 lflow_ref);
     }
 
+    if (has_cr_port) {
+        ds_clear(&match);
+        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
+                      patch_op->cr_port->json_key);
+        ds_clear(&actions);
+        if (od->n_router_ports != od->nbs->n_ports) {
+            ds_put_format(&actions, "clone {outport = %s; output; }; "
+                                    "outport = \""MC_FLOOD_L2"\"; output;",
+                          patch_op->cr_port->json_key);
+            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
+                                    priority, ds_cstr(&match),
+                                    ds_cstr(&actions), stage_hint,
+                                    lflow_ref);
+        } else {
+            ds_put_format(&actions, "outport = %s; output;",
+                          patch_op->cr_port->json_key);
+            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
+                                    priority, ds_cstr(&match),
+                                    ds_cstr(&actions),
+                                    stage_hint,
+                                    lflow_ref);
+        }
+    }
+
+    ds_destroy(&m);
     ds_destroy(&match);
     ds_destroy(&actions);
 }
@@ -9585,7 +9707,11 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
                                 struct ds *actions, struct ds *match)
 {
     ovs_assert(op->nbsp);
-    if (lsp_is_external(op->nbsp)) {
+
+    /* Note: A switch port can also have a chassis resident derived port.
+     * Check if 'op' is a chassis resident dervied port. If so, skip
+     * adding unicast lookup flows for this port. */
+    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
         return;
     }
 
@@ -9603,8 +9729,6 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
                            "outport = \""MC_UNKNOWN "\"; output;"
                          : "outport = %s; output;")
                          : debug_drop_action();
-    ds_clear(actions);
-    ds_put_format(actions, action, op->json_key);
 
     if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
         /* For ports connected to logical routers add flows to bypass the
@@ -9651,14 +9775,43 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
             if (add_chassis_resident_check) {
                 ds_put_format(match, " && is_chassis_resident(%s)", json_key);
             }
+        } else if (op->cr_port) {
+            /* If the op has a chassis resident port, it means
+             *   - its peer is a distributed gateway port (DGP) and
+             *   - routing is centralized for the DGP's networks on
+             *     the configured gateway chassis.
+             *
+             * If that's the case, make sure that the packets destined to
+             * the DGP's MAC are sent to the chassis where the DGP resides.
+             * */
+            ds_clear(actions);
+            ds_put_format(actions, action, op->cr_port->json_key);
+
+            struct ds m = DS_EMPTY_INITIALIZER;
+            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
+                          op->peer->lrp_networks.ea_s,
+                          op->cr_port->json_key);
+
+            ovn_lflow_add_with_hint(lflows, op->od,
+                                    S_SWITCH_IN_L2_LKUP, 50,
+                                    ds_cstr(&m), ds_cstr(actions),
+                                    &op->nbsp->header_,
+                                    op->lflow_ref);
+            ds_destroy(&m);
+            ds_put_format(match, " && is_chassis_resident(%s)",
+                          op->cr_port->json_key);
         }
 
+        ds_clear(actions);
+        ds_put_format(actions, action, op->json_key);
         ovn_lflow_add_with_hint(lflows, op->od,
                                 S_SWITCH_IN_L2_LKUP, 50,
                                 ds_cstr(match), ds_cstr(actions),
                                 &op->nbsp->header_,
                                 op->lflow_ref);
     } else {
+        ds_clear(actions);
+        ds_put_format(actions, action, op->json_key);
         for (size_t i = 0; i < op->n_lsp_addrs; i++) {
             ds_clear(match);
             ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
@@ -11772,6 +11925,14 @@  build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
         return;
     }
 
+    if (op->peer && op->peer->cr_port) {
+        /* We don't add the below flows if the router port's peer has
+         * a chassisresident port.  That's because routing is centralized on
+         * the gateway chassis for the router port networks/subnets.
+         */
+        return;
+    }
+
     /* Mac address to use when replying to ARP/NS. */
     const char *mac_s = REG_INPORT_ETH_ADDR;
     struct eth_addr mac;
@@ -15158,6 +15319,16 @@  lrouter_check_nat_entry(const struct ovn_datapath *od,
     /* For distributed router NAT, determine whether this NAT rule
      * satisfies the conditions for distributed NAT processing. */
     *distributed = false;
+
+    /* NAT cannnot be distributed if the DGP's peer
+     * has a chassisresident port (as the routing is centralized
+     * on the gateway chassis for the DGP's networks/subnets.)
+     */
+    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
+    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
+        return 0;
+    }
+
     if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
         nat->logical_port && nat->external_mac) {
         if (eth_addr_from_string(nat->external_mac, mac)) {
diff --git a/northd/northd.h b/northd/northd.h
index d4a8d75abc..d7c9655916 100644
--- a/northd/northd.h
+++ b/northd/northd.h
@@ -325,6 +325,7 @@  struct ovn_datapath {
      * will be NULL. */
     struct ovn_port **l3dgw_ports;
     size_t n_l3dgw_ports;
+    size_t n_allocated_l3dgw_ports;
 
     /* router datapath has a logical port with redirect-type set to bridged. */
     bool redirect_bridged;
diff --git a/ovn-nb.xml b/ovn-nb.xml
index 9552534f6d..794e2fb961 100644
--- a/ovn-nb.xml
+++ b/ovn-nb.xml
@@ -3451,6 +3451,40 @@  or
           <ref column="options" key="gateway_mtu"/> option.
         </p>
       </column>
+
+      <column name="options" key="centralize_routing"
+              type='{"type": "boolean"}'>
+        <p>
+          This option is applicable only if the router port is a
+          distributed gateway port i.e if the <ref table="Logical_Router_Port"
+          column="ha_chassis_group"/> column or
+          <ref table="Logical_Router_Port" column="gateway_chassis"/>
+          is set.
+        </p>
+
+        <p>
+          If set to <code>true</code>, routing for the router port's
+          networks (set in the column <ref table="Logical_Router_Port"
+          column="networks"/>) is centralized on the gateway chassis
+          which claims this distributed gateway port.
+        </p>
+
+        <p>
+          Additionally for this option to take effect, below conditions
+          must be met:
+        </p>
+
+        <ul>
+          <li>
+            The Logical router has only one distributed gateway port.
+          </li>
+
+          <li>
+            The router port's peer logical switch has no localnet ports.
+          </li>
+
+        </ul>
+      </column>
     </group>
 
     <group title="Attachment">
diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
index ef41087ae3..df99f01b64 100644
--- a/tests/multinode-macros.at
+++ b/tests/multinode-macros.at
@@ -73,7 +73,7 @@  m_count_rows() {
 m_check_row_count() {
     local db=$(parse_db $1) table=$(parse_table $1); shift
     local count=$1; shift
-    local found=$(m_count_rows $c $db:$table "$@")
+    local found=$(m_count_rows $db:$table "$@")
     echo
     echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
     if test "$count" != "$found"; then
diff --git a/tests/multinode.at b/tests/multinode.at
index 1e6eeb6610..9e01a29cc2 100644
--- a/tests/multinode.at
+++ b/tests/multinode.at
@@ -1033,4 +1033,181 @@  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
 done
 M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
 
+# Reset back to geneve tunnels
+for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
+do
+    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
+done
+
+AT_CLEANUP
+
+AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
+
+# Check that ovn-fake-multinode setup is up and running
+check_fake_multinode_setup
+
+# Delete the multinode NB and OVS resources before starting the test.
+cleanup_multinode_resources
+
+check multinode_nbctl ls-add sw0
+check multinode_nbctl lsp-add sw0 sw0-port1
+check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
+check multinode_nbctl lsp-add sw0 sw0-port2
+check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
+
+m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
+m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
+
+m_wait_for_ports_up
+
+M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+# Create the second logical switch with one port
+check multinode_nbctl ls-add sw1
+check multinode_nbctl lsp-add sw1 sw1-port1
+check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
+
+# Create a logical router and attach both logical switches
+check multinode_nbctl lr-add lr0
+check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
+check multinode_nbctl lsp-add sw0 sw0-lr0
+check multinode_nbctl lsp-set-type sw0-lr0 router
+check multinode_nbctl lsp-set-addresses sw0-lr0 router
+check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
+
+check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
+check multinode_nbctl lsp-add sw1 sw1-lr0
+check multinode_nbctl lsp-set-type sw1-lr0 router
+check multinode_nbctl lsp-set-addresses sw1-lr0 router
+check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
+
+m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
+
+# create exteranl connection for N/S traffic
+check multinode_nbctl ls-add public
+check multinode_nbctl lsp-add public ln-public
+check multinode_nbctl lsp-set-type ln-public localnet
+check multinode_nbctl lsp-set-addresses ln-public unknown
+check multinode_nbctl lsp-set-options ln-public network_name=public
+
+check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
+check multinode_nbctl lsp-add public public-lr0
+check multinode_nbctl lsp-set-type public-lr0 router
+check multinode_nbctl lsp-set-addresses public-lr0 router
+check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
+check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
+
+check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
+check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
+check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
+check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
+
+# Create a logical port pub-p1 and bind it in ovn-chassis-1
+check multinode_nbctl lsp-add public public-port1
+check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
+
+m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
+
+check multinode_nbctl --wait=hv sync
+
+# First do basic ping tests before deleting the localnet port - ln-public.
+# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
+# is centralized on ovn-gw-1.
+
+# This function checks the North-South traffic.
+run_ns_traffic() {
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+}
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Delete the localnet port by changing the type of ln-public to VIF port.
+check multinode_nbctl --wait=hv lsp-set-type ln-public ""
+
+# cr-port should not be created for public-lr0 since the option
+# centralize_routing=true is not yet set for lr0-public.
+m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+# Set the option - centralize_routing now.
+check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
+
+m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
+m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Re-add the localnet port
+check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
+
+m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Delete the ln-public port this time.
+check multinode_nbctl --wait=hv lsp-del ln-public
+
+m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
+m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
 AT_CLEANUP
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index a389d19886..5445fff494 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -2188,7 +2188,7 @@  match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
 action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
 ])
 
-# xreg0[0..47] isn't used anywhere else.
+# xreg0[[0..47]] isn't used anywhere else.
 AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
 
 AT_CLEANUP
@@ -5524,13 +5524,14 @@  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
 
 AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
 
-ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
+check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
+check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
 
 ovn-sbctl lflow-list ls1 > ls1_lflows
 AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
   table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
   table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
-  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
   table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
   table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
   table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
@@ -12721,3 +12722,512 @@  AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
 
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD_NO_HV([
+AT_SETUP([NAT on a provider network with no localnet ports])
+AT_KEYWORDS([NAT])
+ovn_start
+
+check ovn-nbctl -- ls-add sw0 -- ls-add sw1
+check ovn-nbctl lsp-add sw0 sw0-port1
+check ovn-nbctl lr-add lr0
+check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
+check ovn-nbctl lsp-add sw0 sw0-lr0
+check ovn-nbctl lsp-set-type sw0-lr0 router
+check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
+check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
+
+check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
+check ovn-nbctl lsp-add sw1 sw1-lr0
+check ovn-nbctl lsp-set-type sw1-lr0 router
+check ovn-nbctl lsp-set-addresses sw1-lr0 router
+check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
+
+check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
+check ovn-nbctl ls-add public
+check ovn-nbctl lsp-add public pub-p1
+
+# localnet port
+check ovn-nbctl lsp-add public ln-public
+check ovn-nbctl lsp-set-type ln-public localnet
+check ovn-nbctl lsp-set-addresses ln-public unknown
+check ovn-nbctl lsp-set-options ln-public network_name=public
+
+check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
+check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
+
+check ovn-nbctl lsp-add public public-lr0
+check ovn-nbctl lsp-set-type public-lr0 router
+check ovn-nbctl lsp-set-addresses public-lr0 router
+check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
+
+check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
+check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
+check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
+check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
+
+check ovn-nbctl --wait=sb sync
+
+check_flows_no_cr_port_for_public_lr0() {
+  # check that there is no port binding cr-public-lr0
+  check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+  ovn-sbctl dump-flows lr0 > lr0flows
+  ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
+  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
+  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+])
+
+AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
+  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
+  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+])
+
+AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+])
+
+AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+])
+
+AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
+])
+
+AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+])
+
+AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+])
+
+AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
+  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+}
+
+check_flows_cr_port_for_public_lr0() {
+  # check that there is port binding cr-public-lr0
+  check_row_count Port_Binding 1 logical_port=cr-public-lr0
+  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+  ovn-sbctl dump-flows lr0 > lr0flows
+  ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
+  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
+  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+])
+
+AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
+  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
+  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+])
+
+AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+])
+
+AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+])
+
+AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
+])
+
+AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+])
+
+AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+])
+
+AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
+  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+}
+
+# Check that the lflows are as expected when public has localnet port.
+check_flows_no_cr_port_for_public_lr0
+
+# Remove the localnet port from public logical switch.
+check ovn-nbctl --wait=sb lsp-set-type ln-public ""
+
+# Check that the lflows are as expected and there is no cr port
+# created for "public-lr0"  when public has no localnet port
+# since public doesn't have the option "overlay_provider_network=true"
+# set.
+check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+ovn-sbctl dump-flows lr0 > lr0flows
+ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+
+# Set the option "centralize_routing=true" for lr0-public.
+check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
+
+# Check that the lflows are as expected and there is cr port created for public-lr0.
+check_flows_cr_port_for_public_lr0
+
+# Set the type of ln-public back to localnet
+check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
+
+# Check that the lflows are as expected when public has localnet port.
+check_flows_no_cr_port_for_public_lr0
+
+# Delete the localnet port
+check ovn-nbctl --wait=sb lsp-del ln-public
+
+# Check that the lflows are as expected when public has no localnet port.
+check_flows_cr_port_for_public_lr0
+
+# Create multiple gateway ports.  chassisresident port should not be
+# created for 'public-lr0' even if there is no localnet port on 'public'
+# logical switch.
+check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
+# check that there is no port binding cr-public-lr0
+check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+ovn-sbctl dump-flows lr0 > lr0flows
+ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CLEANUP
+])
diff --git a/tests/ovn.at b/tests/ovn.at
index 185ba4a21e..21d7484f9e 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -21232,10 +21232,10 @@  ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
     type=router options:router-port=sw0 \
     -- lsp-set-addresses rp-sw0 router
 
-ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
-    -- lrp-set-gateway-chassis sw1 hv2
+ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
+    -- lrp-set-gateway-chassis lr0-sw1 hv2
 ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
-    type=router options:router-port=sw1 \
+    type=router options:router-port=lr0-sw1 \
     -- lsp-set-addresses rp-sw1 router
 
 ovn-nbctl lsp-add sw0 sw0-p0 \
@@ -21247,6 +21247,8 @@  ovn-nbctl lsp-add sw0 sw0-p1 \
 ovn-nbctl lsp-add sw1 sw1-p0 \
     -- lsp-set-addresses sw1-p0 unknown
 
+check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
+
 ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
 ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64