| Message ID | 20210619095124.3596487-4-mark.d.gray@redhat.com |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | northd: Refactor Logical Flows for Gateway Router with DNAT/Load Balancers | expand |
On Sat, Jun 19, 2021 at 2:52 AM Mark Gray <mark.d.gray@redhat.com> wrote: > > This patch addresses a number of interconnected issues with Gateway Routers > that have Load Balancing enabled: > > 1) In the router pipeline, we have the following stages to handle > dnat and unsnat. > > - Stage 4 : lr_in_defrag (dnat zone) > - Stage 5 : lr_in_unsnat (snat zone) > - Stage 6 : lr_in_dnat (dnat zone) > > In the reply direction, the order of traversal of the tables > "lr_in_defrag", "lr_in_unsnat" and "lr_in_dnat" adds incorrect > datapath flows that check ct_state in the wrong conntrack zone. > This is illustrated below where reply trafic enters the physical host > port (6) and traverses DNAT zone (14), SNAT zone (default), back to the > DNAT zone and then on to Logical Switch Port zone (22). The third > flow is incorrectly checking the state from the SNAT zone instead > of the DNAT zone. > > recirc_id(0),in_port(6),ct_state(-new-est-rel-rpl-trk) actions:ct_clear,ct(zone=14),recirc(0xf) > recirc_id(0xf),in_port(6) actions:ct(nat),recirc(0x10) > recirc_id(0x10),in_port(6),ct_state(-new+est+trk) actions:ct(zone=14,nat),recirc(0x11) > recirc_id(0x11),in_port(6),ct_state(+new-est-rel-rpl+trk) actions: ct(zone=22,nat),recirc(0x12) > recirc_id(0x12),in_port(6),ct_state(-new+est-rel+rpl+trk) actions:5 > > Update the order of these tables to resolve this. > > 2) Efficiencies can be gained by using the ct_dnat action in the > table "lr_in_defrag" instead of ct_next. This removes the need for the > ct_dnat action for established Load Balancer flows avoiding a > recirculation. > > 3) On a Gateway router with DNAT flows configured, the router will translate > the destination IP address from (A) to (B). Reply packets from (B) are > correctly UNDNATed in the reverse direction. > > However, if a new connection is established from (B), this flow is never > committed to conntrack and, as such, is never established. This will > cause OVS datapath flows to be added that match on the ct.new flag. > > For software-only datapaths this is not a problem. However, for > datapaths that offload these flows to hardware, this may be problematic > as some devices are unable to offload flows that match on ct.new. > > This patch resolves this by committing these flows to the DNAT zone in > the new "lr_out_post_undnat" stage. Although this could be done in the > DNAT zone, by doing this in the new zone we can avoid a recirculation. > > Co-authored-by: Numan Siddique <numans@ovn.org> > Signed-off-by: Mark Gray <mark.d.gray@redhat.com> > Signed-off-by: Numan Siddique <numans@ovn.org> Thanks Mark and Numan. Please see some non-critical comments inlined. > --- > northd/ovn-northd.8.xml | 143 +++++---- > northd/ovn-northd.c | 111 +++++-- > northd/ovn_northd.dl | 113 +++++-- > tests/ovn-northd.at | 674 +++++++++++++++++++++++++++++++++++----- > tests/ovn.at | 6 +- > tests/system-ovn.at | 13 +- > 6 files changed, 853 insertions(+), 207 deletions(-) > > diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml > index 4074646029b4..d56a121d4d2e 100644 > --- a/northd/ovn-northd.8.xml > +++ b/northd/ovn-northd.8.xml > @@ -2628,39 +2628,9 @@ icmp6 { > </li> > </ul> > > - <h3>Ingress Table 4: DEFRAG</h3> > > - <p> > - This is to send packets to connection tracker for tracking and > - defragmentation. It contains a priority-0 flow that simply moves traffic > - to the next table. > - </p> > - > - <p> > - If load balancing rules with virtual IP addresses (and ports) are > - configured in <code>OVN_Northbound</code> database for a Gateway router, > - a priority-100 flow is added for each configured virtual IP address > - <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip > - && ip4.dst == <var>VIP</var></code>. For IPv6 <var>VIPs</var>, > - the flow matches <code>ip && ip6.dst == <var>VIP</var></code>. > - The flow uses the action <code>ct_next;</code> to send IP packets to the > - connection tracker for packet de-fragmentation and tracking before > - sending it to the next table. > - </p> > - > - <p> > - If ECMP routes with symmetric reply are configured in the > - <code>OVN_Northbound</code> database for a gateway router, a priority-300 > - flow is added for each router port on which symmetric replies are > - configured. The matching logic for these ports essentially reverses the > - configured logic of the ECMP route. So for instance, a route with a > - destination routing policy will instead match if the source IP address > - matches the static route's prefix. The flow uses the action > - <code>ct_next</code> to send IP packets to the connection tracker for > - packet de-fragmentation and tracking before sending it to the next table. > - </p> > > - <h3>Ingress Table 5: UNSNAT</h3> > + <h3>Ingress Table 4: UNSNAT</h3> > > <p> > This is for already established connections' reverse traffic. > @@ -2669,7 +2639,7 @@ icmp6 { > unSNATted here. > </p> > > - <p>Ingress Table 5: UNSNAT on Gateway and Distributed Routers</p> > + <p>Ingress Table 4: UNSNAT on Gateway and Distributed Routers</p> > <ul> > <li> > <p> > @@ -2696,7 +2666,7 @@ icmp6 { > </li> > </ul> > > - <p>Ingress Table 5: UNSNAT on Gateway Routers</p> > + <p>Ingress Table 4: UNSNAT on Gateway Routers</p> > > <ul> > <li> > @@ -2713,9 +2683,10 @@ icmp6 { > <code>lb_force_snat_ip=router_ip</code> then for every logical router > port <var>P</var> attached to the Gateway router with the router ip > <var>B</var>, a priority-110 flow is added with the match > - <code>inport == <var>P</var> && ip4.dst == <var>B</var></code> or > - <code>inport == <var>P</var> && ip6.dst == <var>B</var></code> > - with an action <code>ct_snat; </code>. > + <code>inport == <var>P</var> && > + ip4.dst == <var>B</var></code> or <code>inport == <var>P</var> > + && ip6.dst == <var>B</var></code> with an action > + <code>ct_snat; </code>. > </p> > > <p> > @@ -2745,7 +2716,7 @@ icmp6 { > </li> > </ul> > > - <p>Ingress Table 5: UNSNAT on Distributed Routers</p> > + <p>Ingress Table 4: UNSNAT on Distributed Routers</p> > > <ul> > <li> > @@ -2776,6 +2747,40 @@ icmp6 { > </li> > </ul> > > + <h3>Ingress Table 5: DEFRAG</h3> > + > + <p> > + This is to send packets to connection tracker for tracking and > + defragmentation. It contains a priority-0 flow that simply moves traffic > + to the next table. > + </p> > + > + <p> > + If load balancing rules with virtual IP addresses (and ports) are > + configured in <code>OVN_Northbound</code> database for a Gateway router, > + a priority-100 flow is added for each configured virtual IP address > + <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip > + && ip4.dst == <var>VIP</var></code>. For IPv6 <var>VIPs</var>, > + the flow matches <code>ip && ip6.dst == <var>VIP</var></code>. > + The flow applies the action <code>reg0 = <var>VIP</var> nit: for IPv6 it is xxreg0. > + && ct_dnat;</code> to send IP packets to the > + connection tracker for packet de-fragmentation and to dnat the > + destination IP for the committed connection before sending it to the > + next table. > + </p> > + > + <p> > + If ECMP routes with symmetric reply are configured in the > + <code>OVN_Northbound</code> database for a gateway router, a priority-300 > + flow is added for each router port on which symmetric replies are > + configured. The matching logic for these ports essentially reverses the > + configured logic of the ECMP route. So for instance, a route with a > + destination routing policy will instead match if the source IP address > + matches the static route's prefix. The flow uses the action > + <code>ct_next</code> to send IP packets to the connection tracker for > + packet de-fragmentation and tracking before sending it to the next table. > + </p> > + > <h3>Ingress Table 6: DNAT</h3> > > <p> > @@ -2828,19 +2833,28 @@ icmp6 { > </li> > > <li> > - For all the configured load balancing rules for a router in > - <code>OVN_Northbound</code> database that includes a L4 port > - <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 address > - <var>VIP</var>, a priority-120 flow that matches on > - <code>ct.est && ip && ip4.dst == <var>VIP</var> > - && <var>P</var> && <var>P</var>.dst == <var>PORT > - </var></code> (<code>ip6.dst == <var>VIP</var></code> in the IPv6 case) > - with an action of <code>ct_dnat;</code>. If the router is > - configured to force SNAT any load-balanced packets, the above action > - will be replaced by <code>flags.force_snat_for_lb = 1; ct_dnat;</code>. > - If the load balancing rule is configured with <code>skip_snat</code> > - set to true, the above action will be replaced by > - <code>flags.skip_snat_for_lb = 1; ct_dnat;</code>. > + <p> > + For all the configured load balancing rules for a router in > + <code>OVN_Northbound</code> database that includes a L4 port > + <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 address > + <var>VIP</var>, a priority-120 flow that matches on > + <code>ct.est && ip && reg0 == <var>VIP</var> > + && <var>P</var> && <var>P</var>.dst == <var>PORT > + </var></code> (<code>xxreg0 == <var>VIP</var></code> in the > + IPv6 case) with an action of <code>next;</code>. If the router is > + configured to force SNAT any load-balanced packets, the above action > + will be replaced by <code>flags.force_snat_for_lb = 1; next;</code>. > + If the load balancing rule is configured with <code>skip_snat</code> > + set to true, the above action will be replaced by > + <code>flags.skip_snat_for_lb = 1; next;</code>. > + </p> > + > + <p> > + Previous table <code>lr_in_defrag</code> sets the register > + <code>reg0</code> (or <code>xxreg0</code> for IPv6) and does > + <code>ct_dnat</code>. Hence for established traffic, this > + table just advances the packet to the next stage. > + </p> > </li> > In this section "Ingress Table 6: DNAT", there are several paragraphs with very similar text but only minor differences on the match condition, such as "includes just an IP address ...", and you need to update for all those paragraphs. I understand it is painful to update the redundant information, and it may be also painful for the readers, but it seems even worse if it is incorrect. Not sure if there is a better way to maintain this document. (the DDlog code is easier to understand than the document) > <li> > @@ -3876,7 +3890,26 @@ nd_ns { > </li> > </ul> > > - <h3>Egress Table 1: SNAT</h3> > + <h3>Egress Table 1: Post UNDNAT on Gateway Routers</h3> > + > + <p> > + <ul> > + <li> > + A priority-50 logical flow is added that commits any untracked flows > + from the previous table <code>lr_out_undnat</code>. This flow > + matches on <code>ct.new && ip</code> with action > + <code>ct_commit { } ; next; </code>. > + </li> > + > + <li> > + A priority-0 logical flow with match <code>1</code> has actions > + <code>next;</code>. > + </li> > + > + </ul> > + </p> > + > + <h3>Egress Table 2: SNAT</h3> > > <p> > Packets that are configured to be SNATed get their source IP address > @@ -3892,7 +3925,7 @@ nd_ns { > </li> > </ul> > > - <p>Egress Table 1: SNAT on Gateway Routers</p> > + <p>Egress Table 2: SNAT on Gateway Routers</p> > > <ul> > <li> > @@ -3991,7 +4024,7 @@ nd_ns { > </li> > </ul> > > - <p>Egress Table 1: SNAT on Distributed Routers</p> > + <p>Egress Table 2: SNAT on Distributed Routers</p> > > <ul> > <li> > @@ -4051,7 +4084,7 @@ nd_ns { > </li> > </ul> > > - <h3>Egress Table 2: Egress Loopback</h3> > + <h3>Egress Table 3: Egress Loopback</h3> > > <p> > For distributed logical routers where one of the logical router > @@ -4120,7 +4153,7 @@ clone { > </li> > </ul> > > - <h3>Egress Table 3: Delivery</h3> > + <h3>Egress Table 4: Delivery</h3> > > <p> > Packets that reach this table are ready for delivery. It contains: > diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c > index d97ab4a5b39c..27e5fbea9f4f 100644 > --- a/northd/ovn-northd.c > +++ b/northd/ovn-northd.c > @@ -187,8 +187,8 @@ enum ovn_stage { > PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ > PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ > PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ > - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 4, "lr_in_defrag") \ > - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 5, "lr_in_unsnat") \ > + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ > + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ > PIPELINE_STAGE(ROUTER, IN, DNAT, 6, "lr_in_dnat") \ > PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, "lr_in_ecmp_stateful") \ > PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, "lr_in_nd_ra_options") \ > @@ -204,10 +204,11 @@ enum ovn_stage { > PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 18, "lr_in_arp_request") \ > \ > /* Logical router egress stages. */ \ > - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ > - PIPELINE_STAGE(ROUTER, OUT, SNAT, 1, "lr_out_snat") \ > - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 2, "lr_out_egr_loop") \ > - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 3, "lr_out_delivery") > + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ > + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 1, "lr_out_post_undnat") \ > + PIPELINE_STAGE(ROUTER, OUT, SNAT, 2, "lr_out_snat") \ > + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 3, "lr_out_egr_loop") \ > + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 4, "lr_out_delivery") > > #define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ > S_##DP_TYPE##_##PIPELINE##_##STAGE \ > @@ -643,6 +644,12 @@ struct ovn_datapath { > /* Multicast data. */ > struct mcast_info mcast_info; > > + /* Applies to only logical router datapath. > + * True if logical router is a gateway router. i.e options:chassis is set. > + * If this is true, then 'l3dgw_port' and 'l3redirect_port' will be > + * ignored. */ > + bool is_gw_router; > + This looks redundant with the field l3dgw_port. The newly added bool is used only once, and the other places are still using !l3dgw_port to check if it is GW router. I suggest either keep using !l3dgw_port or unifying all the checks to use the new bool. > /* OVN northd only needs to know about the logical router gateway port for > * NAT on a distributed router. This "distributed gateway port" is > * populated only when there is a gateway chassis specified for one of > @@ -1247,6 +1254,9 @@ join_datapaths(struct northd_context *ctx, struct hmap *datapaths, > init_mcast_info_for_datapath(od); > init_nat_entries(od); > init_lb_ips(od); > + if (smap_get(&od->nbr->options, "chassis")) { > + od->is_gw_router = true; > + } > ovs_list_push_back(lr_list, &od->lr_list); > } > } > @@ -8731,20 +8741,33 @@ add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od, > } > > /* A match and actions for established connections. */ > - char *est_match = xasprintf("ct.est && %s", ds_cstr(match)); > + struct ds est_match = DS_EMPTY_INITIALIZER; > + ds_put_format(&est_match, > + "ct.est && ip && %sreg0 == %s && ct_label.natted == 1", > + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) ? "" : "xx", > + lb_vip->vip_str); > + if (lb_vip->vip_port) { > + ds_put_format(&est_match, " && %s", proto); > + } > + if (od->l3redirect_port && > + (lb_vip->n_backends || !lb_vip->empty_backend_rej)) { > + ds_put_format(&est_match, " && is_chassis_resident(%s)", > + od->l3redirect_port->json_key); > + } This part is a little redundant with constructing the "match" in the caller of this function. The only difference is matching ip.dst or "reg0/xxreg0". It is not anything critical but it may be better to put the logic at the same place, maybe just move the logic from the caller to this function. > if (snat_type == FORCE_SNAT || snat_type == SKIP_SNAT) { > - char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; ct_dnat;", > + char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; next;", > snat_type == SKIP_SNAT ? "skip" : "force"); > ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, > - est_match, est_actions, &lb->header_); > + ds_cstr(&est_match), est_actions, > + &lb->header_); > free(est_actions); > } else { > ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, > - est_match, "ct_dnat;", &lb->header_); > + ds_cstr(&est_match), "next;", &lb->header_); > } > > free(new_match); > - free(est_match); > + ds_destroy(&est_match); > > const char *ip_match = NULL; > if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { > @@ -8829,8 +8852,8 @@ add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od, > static void > build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, > struct hmap *lbs, struct shash *meter_groups, > - struct sset *nat_entries, struct ds *match, > - struct ds *actions) > + struct sset *nat_entries, > + struct ds *match, struct ds *actions) > { > /* A set to hold all ips that need defragmentation and tracking. */ > struct sset all_ips = SSET_INITIALIZER(&all_ips); > @@ -8852,10 +8875,17 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, > for (size_t j = 0; j < lb->n_vips; j++) { > struct ovn_lb_vip *lb_vip = &lb->vips[j]; > struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[j]; > + > + bool is_udp = nullable_string_is_equal(nb_lb->protocol, "udp"); > + bool is_sctp = nullable_string_is_equal(nb_lb->protocol, > + "sctp"); > + const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : "tcp"; > + > ds_clear(actions); > build_lb_vip_actions(lb_vip, lb_vip_nb, actions, > lb->selection_fields, false); > > + struct ds defrag_actions = DS_EMPTY_INITIALIZER; > if (!sset_contains(&all_ips, lb_vip->vip_str)) { > sset_add(&all_ips, lb_vip->vip_str); > /* If there are any load balancing rules, we should send > @@ -8867,17 +8897,28 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, > * 2. If there are L4 ports in load balancing rules, we > * need the defragmentation to match on L4 ports. */ > ds_clear(match); > + ds_clear(&defrag_actions); > if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { > ds_put_format(match, "ip && ip4.dst == %s", > lb_vip->vip_str); > + ds_put_format(&defrag_actions, "reg0 = %s; ct_dnat;", > + lb_vip->vip_str); > } else { > ds_put_format(match, "ip && ip6.dst == %s", > lb_vip->vip_str); > + ds_put_format(&defrag_actions, "xxreg0 = %s; ct_dnat;", > + lb_vip->vip_str); > + } > + > + if (lb_vip->vip_port) { > + ds_put_format(match, " && %s", proto); > } > ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, > - 100, ds_cstr(match), "ct_next;", > + 100, ds_cstr(match), > + ds_cstr(&defrag_actions), > &nb_lb->header_); > } > + ds_destroy(&defrag_actions); > > /* Higher priority rules are added for load-balancing in DNAT > * table. For every match (on a VIP[:port]), we add two flows > @@ -8886,18 +8927,14 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, > * flow is for ct.est with an action of "ct_dnat;". */ > ds_clear(match); > if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { > - ds_put_format(match, "ip && ip4.dst == %s", > + ds_put_format(match, "ip && reg0 == %s", > lb_vip->vip_str); > } else { > - ds_put_format(match, "ip && ip6.dst == %s", > + ds_put_format(match, "ip && xxreg0 == %s", > lb_vip->vip_str); > } > > int prio = 110; > - bool is_udp = nullable_string_is_equal(nb_lb->protocol, "udp"); > - bool is_sctp = nullable_string_is_equal(nb_lb->protocol, > - "sctp"); > - const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : "tcp"; > > if (lb_vip->vip_port) { > ds_put_format(match, " && %s && %s.dst == %d", proto, > @@ -11400,8 +11437,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, > * part of a reply. We undo the DNAT here. > * > * Note that this only applies for NAT on a distributed router. > - * Undo DNAT on a gateway router is done in the ingress DNAT > - * pipeline stage. */ > + */ > if (!od->l3dgw_port || > (strcmp(nat->type, "dnat") && strcmp(nat->type, "dnat_and_snat"))) { > return; > @@ -11681,9 +11717,22 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, > ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); > ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); > ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); > + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); > ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); > ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); > > + /* For Gateway routers, if the gateway router has load balancer or DNAT > + * rules, we commit newly initiated connections in the reply direction > + * to the DNAT zone. This ensures that these flows are tracked. If the flow > + * was not committed, it would produce ongoing datapath flows with the > + * ct.new flag set. Some NICs are unable to offload these flows. > + */ > + if (od->is_gw_router && > + (od->nbr->n_nat || od->nbr->n_load_balancer)) { > + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, > + "ip && ct.new", "ct_commit { } ; next; "); > + } > + Why do the commit for GW routers ONLY? I think it may be better to keep the behavior consistent for both distributed routers and GW routers. I understand that distributed routers has flows that match each individual backends IP [&& port], so there are less chances to go through this stage, but it is still possible if a backend initiates a connection to external (when L4 port is not specified for the VIP, or the source port happens to be the VIP's port). And it seems not harmful to add the flow for distributed routers if only very few connections would hit this flow. Moreover, I think it is better to keep the behavior consistent between the two types of routers also regarding the individual backends IP [&& port] check. There were control plane scale concerns discussed. However, from what I observed with DP group feature enabled, the impact should be minimal. So I'd suggest introducing the same checks for GW router datapaths before going through the UNDNAT stage, to avoid the extra recirculation for most use cases. It would be great to have a knob to turn it off when there is scale concerns. However, please feel free to add it as a follow-up patch if it is ok. > /* Send the IPv6 NS packets to next table. When ovn-controller > * generates IPv6 NS (for the action - nd_ns{}), the injected > * packet would go through conntrack - which is not required. */ > @@ -11848,18 +11897,12 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, > od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); > } > } > - > - /* For gateway router, re-circulate every packet through > - * the DNAT zone. This helps with the following. > - * > - * Any packet that needs to be unDNATed in the reverse > - * direction gets unDNATed. Ideally this could be done in > - * the egress pipeline. But since the gateway router > - * does not have any feature that depends on the source > - * ip address being external IP address for IP routing, > - * we can do it here, saving a future re-circulation. */ > - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, > - "ip", "flags.loopback = 1; ct_dnat;"); > + /* For gateway router, re-circulate every packet through the DNAT zone > + * so that packets that need to be unDNATed in the reverse direction > + * get unDNATed. > + */ > + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, > + "ip", "flags.loopback = 1; ct_dnat;"); This change looks reasonable to me, because I don't understand the comment above regarding how doing the UNDNAT in the ROUTER_IN_DNAT stage would save a recirculation. Could you update this part in the ovn-northd.xml document to reflect this change as well? It is still mentioned in the document "For Gateway routers, the unDNAT processing is carried out in the ingress DNAT table." Thanks, Han > } > > /* Load balancing and packet defrag are only valid on > diff --git a/northd/ovn_northd.dl b/northd/ovn_northd.dl > index 3afa80a3b549..727641ac6ae1 100644 > --- a/northd/ovn_northd.dl > +++ b/northd/ovn_northd.dl > @@ -1454,8 +1454,8 @@ function s_ROUTER_IN_ADMISSION(): Stage { Stage{Ingress, 0, "lr_in_admiss > function s_ROUTER_IN_LOOKUP_NEIGHBOR(): Stage { Stage{Ingress, 1, "lr_in_lookup_neighbor"} } > function s_ROUTER_IN_LEARN_NEIGHBOR(): Stage { Stage{Ingress, 2, "lr_in_learn_neighbor"} } > function s_ROUTER_IN_IP_INPUT(): Stage { Stage{Ingress, 3, "lr_in_ip_input"} } > -function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 4, "lr_in_defrag"} } > -function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 5, "lr_in_unsnat"} } > +function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 4, "lr_in_unsnat"} } > +function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 5, "lr_in_defrag"} } > function s_ROUTER_IN_DNAT(): Stage { Stage{Ingress, 6, "lr_in_dnat"} } > function s_ROUTER_IN_ECMP_STATEFUL(): Stage { Stage{Ingress, 7, "lr_in_ecmp_stateful"} } > function s_ROUTER_IN_ND_RA_OPTIONS(): Stage { Stage{Ingress, 8, "lr_in_nd_ra_options"} } > @@ -1472,9 +1472,10 @@ function s_ROUTER_IN_ARP_REQUEST(): Stage { Stage{Ingress, 18, "lr_in_arp_re > > /* Logical router egress stages. */ > function s_ROUTER_OUT_UNDNAT(): Stage { Stage{ Egress, 0, "lr_out_undnat"} } > -function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 1, "lr_out_snat"} } > -function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 2, "lr_out_egr_loop"} } > -function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 3, "lr_out_delivery"} } > +function s_ROUTER_OUT_POST_UNDNAT(): Stage { Stage{ Egress, 1, "lr_out_post_undnat"} } > +function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 2, "lr_out_snat"} } > +function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 3, "lr_out_egr_loop"} } > +function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 4, "lr_out_delivery"} } > > /* > * OVS register usage: > @@ -2886,7 +2887,8 @@ for (&Switch(._uuid = ls_uuid)) { > function get_match_for_lb_key(ip_address: v46_ip, > port: bit<16>, > protocol: Option<string>, > - redundancy: bool): string = { > + redundancy: bool, > + use_nexthop_reg: bool): string = { > var port_match = if (port != 0) { > var proto = if (protocol == Some{"udp"}) { > "udp" > @@ -2900,8 +2902,18 @@ function get_match_for_lb_key(ip_address: v46_ip, > }; > > var ip_match = match (ip_address) { > - IPv4{ipv4} -> "ip4.dst == ${ipv4}", > - IPv6{ipv6} -> "ip6.dst == ${ipv6}" > + IPv4{ipv4} -> > + if (use_nexthop_reg) { > + "${rEG_NEXT_HOP()} == ${ipv4}" > + } else { > + "ip4.dst == ${ipv4}" > + }, > + IPv6{ipv6} -> > + if (use_nexthop_reg) { > + "xx${rEG_NEXT_HOP()} == ${ipv6}" > + } else { > + "ip6.dst == ${ipv6}" > + } > }; > > if (redundancy) { "ip && " } else { "" } ++ ip_match ++ port_match > @@ -2935,7 +2947,11 @@ function build_lb_vip_actions(lbvip: Intern<LBVIPWithStatus>, > for (pair in lbvip.backends) { > (var backend, var up) = pair; > if (up) { > - up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") > + if (backend.port != 0) { > + up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") > + } else { > + up_backends.insert("${backend.ip.to_bracketed_string()}") > + } > } > }; > > @@ -2981,7 +2997,7 @@ Flow(.logical_datapath = sw._uuid, > > build_lb_vip_actions(lbvip, s_SWITCH_OUT_QOS_MARK(), actions0 ++ actions1) > }, > - var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, false). > + var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, false, false). > > /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). > * Packets that don't need hairpinning should continue processing. > @@ -3019,7 +3035,7 @@ for (&Switch(._uuid = ls_uuid, .has_lb_vip = true)) { > .__match = "ip && ct.new && ct.trk && ${rEGBIT_HAIRPIN()} == 1", > .actions = "ct_snat_to_vip; next;", > .external_ids = stage_hint(ls_uuid)); > - > + > /* If packet needs to be hairpinned, for established sessions there > * should already be an SNAT conntrack entry. > */ > @@ -5379,13 +5395,14 @@ function default_allow_flow(datapath: uuid, stage: Stage): Flow { > .actions = "next;", > .external_ids = map_empty()} > } > -for (&Router(._uuid = lr_uuid)) { > +for (r in &Router(._uuid = lr_uuid)) { > /* Packets are allowed by default. */ > Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DEFRAG())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_UNSNAT())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_SNAT())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DNAT())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_UNDNAT())]; > + Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_POST_UNDNAT())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_EGR_LOOP())]; > Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_ECMP_STATEFUL())]; > > @@ -5400,6 +5417,25 @@ for (&Router(._uuid = lr_uuid)) { > .external_ids = map_empty()) > } > > +for (r in &Router(._uuid = lr_uuid, > + .is_gateway = is_gateway, > + .nat = nat, > + .load_balancer = load_balancer) > + if is_gateway and (not is_empty(nat) or not is_empty(load_balancer))) { > + /* For Gateway routers, if the gateway router has load balancer or DNAT > + * rules, we commit newly initiated connections in the reply direction > + * to the DNAT zone. This ensures that these flows are tracked. If the flow > + * was not committed, it would produce ongoing datapath flows with the > + * ct.new flag set. Some NICs are unable to offload these flows. > + */ > + Flow(.logical_datapath = lr_uuid, > + .stage = s_ROUTER_OUT_POST_UNDNAT(), > + .priority = 50, > + .__match = "ip && ct.new", > + .actions = "ct_commit { } ; next; ", > + .external_ids = map_empty()) > +} > + > Flow(.logical_datapath = lr, > .stage = s_ROUTER_OUT_SNAT(), > .priority = 120, > @@ -5438,7 +5474,7 @@ function lrouter_nat_add_ext_ip_match( > Some{AllowedExtIps{__as}} -> (" && ${ipX}.${dir} == $${__as.name}", None), > Some{ExemptedExtIps{__as}} -> { > /* Priority of logical flows corresponding to exempted_ext_ips is > - * +1 of the corresponding regulr NAT rule. > + * +1 of the corresponding regular NAT rule. > * For example, if we have following NAT rule and we associate > * exempted external ips to it: > * "ovn-nbctl lr-nat-add router dnat_and_snat 10.15.24.139 50.0.0.11" > @@ -5746,8 +5782,7 @@ for (r in &Router(._uuid = lr_uuid, > * part of a reply. We undo the DNAT here. > * > * Note that this only applies for NAT on a distributed router. > - * Undo DNAT on a gateway router is done in the ingress DNAT > - * pipeline stage. */ > + */ > if ((nat.nat.__type == "dnat" or nat.nat.__type == "dnat_and_snat")) { > Some{var gwport} = l3dgw_port in > var __match = > @@ -5953,16 +5988,11 @@ for (r in &Router(._uuid = lr_uuid, > .context = "lb"); > > /* For gateway router, re-circulate every packet through > - * the DNAT zone. This helps with the following. > - * > - * Any packet that needs to be unDNATed in the reverse > - * direction gets unDNATed. Ideally this could be done in > - * the egress pipeline. But since the gateway router > - * does not have any feature that depends on the source > - * ip address being external IP address for IP routing, > - * we can do it here, saving a future re-circulation. */ > + * the DNAT zone so that packets that need to be unDNATed in the reverse > + * direction get unDNATed. > + */ > Flow(.logical_datapath = lr_uuid, > - .stage = s_ROUTER_IN_DNAT(), > + .stage = s_ROUTER_OUT_UNDNAT(), > .priority = 50, > .__match = "ip", > .actions = "flags.loopback = 1; ct_dnat;", > @@ -6024,7 +6054,16 @@ for (RouterLBVIP( > * pick a DNAT ip address from a group. > * 2. If there are L4 ports in load balancing rules, we > * need the defragmentation to match on L4 ports. */ > - var __match = "ip && ${ipX}.dst == ${ip_address}" in > + var match1 = "ip && ${ipX}.dst == ${ip_address}" in > + var match2 = > + if (port != 0) { > + " && ${proto}" > + } else { > + "" > + } in > + var __match = match1 ++ match2 in > + var xx = ip_address.xxreg() in > + var __actions = "${xx}${rEG_NEXT_HOP()} = ${ip_address}; ct_dnat;" in > /* One of these flows must be created for each unique LB VIP address. > * We create one for each VIP:port pair; flows with the same IP and > * different port numbers will produce identical flows that will > @@ -6033,7 +6072,7 @@ for (RouterLBVIP( > .stage = s_ROUTER_IN_DEFRAG(), > .priority = 100, > .__match = __match, > - .actions = "ct_next;", > + .actions = __actions, > .external_ids = stage_hint(lb._uuid)); > > /* Higher priority rules are added for load-balancing in DNAT > @@ -6041,7 +6080,8 @@ for (RouterLBVIP( > * via add_router_lb_flow(). One flow is for specific matching > * on ct.new with an action of "ct_lb($targets);". The other > * flow is for ct.est with an action of "ct_dnat;". */ > - var match1 = "ip && ${ipX}.dst == ${ip_address}" in > + var xx = ip_address.xxreg() in > + var match1 = "ip && ${xx}${rEG_NEXT_HOP()} == ${ip_address}" in > (var prio, var match2) = > if (port != 0) { > (120, " && ${proto} && ${proto}.dst == ${port}") > @@ -6056,12 +6096,21 @@ for (RouterLBVIP( > var snat_for_lb = snat_for_lb(r.options, lb) in > { > /* A match and actions for established connections. */ > - var est_match = "ct.est && " ++ __match in > + var est_match = "ct.est && " ++ match1 ++ " && ct_label.natted == 1" ++ > + if (port != 0) { > + " && ${proto}" > + } else { > + "" > + } ++ > + match ((l3dgw_port, backends != "" or lb.options.get_bool_def("reject", false))) { > + (Some{gwport}, true) -> " && is_chassis_resident(${redirect_port_name})", > + _ -> "" > + } in > var actions = > match (snat_for_lb) { > - SkipSNAT -> "flags.skip_snat_for_lb = 1; ct_dnat;", > - ForceSNAT -> "flags.force_snat_for_lb = 1; ct_dnat;", > - _ -> "ct_dnat;" > + SkipSNAT -> "flags.skip_snat_for_lb = 1; next;", > + ForceSNAT -> "flags.force_snat_for_lb = 1; next;", > + _ -> "next;" > } in > Flow(.logical_datapath = lr_uuid, > .stage = s_ROUTER_IN_DNAT(), > @@ -6152,7 +6201,7 @@ Flow(.logical_datapath = r._uuid, > r.load_balancer.contains(lb._uuid), > var __match > = "ct.new && " ++ > - get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, true) ++ > + get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, true, true) ++ > match (r.l3dgw_port) { > Some{gwport} -> " && is_chassis_resident(${r.redirect_port_name})", > _ -> "" > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > index d81975cb18a6..accaa87033f4 100644 > --- a/tests/ovn-northd.at > +++ b/tests/ovn-northd.at > @@ -1406,40 +1406,39 @@ AT_SETUP([ovn -- Load balancer VIP in NAT entries]) > AT_SKIP_IF([test $HAVE_PYTHON = no]) > ovn_start > > -ovn-nbctl lr-add lr0 > -ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 > -ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 > +check ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 > > -ovn-nbctl set logical_router lr0 options:chassis=ch1 > +check ovn-nbctl set logical_router lr0 options:chassis=ch1 > > -ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" > -ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp > -ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" > -ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" > +check ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" > +check ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp > +check ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" > +check ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" > > -ovn-nbctl lr-lb-add lr0 lb1 > -ovn-nbctl lr-lb-add lr0 lb2 > -ovn-nbctl lr-lb-add lr0 lb3 > -ovn-nbctl lr-lb-add lr0 lb4 > +check ovn-nbctl lr-lb-add lr0 lb1 > +check ovn-nbctl lr-lb-add lr0 lb2 > +check ovn-nbctl lr-lb-add lr0 lb3 > +check ovn-nbctl lr-lb-add lr0 lb4 > > -ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 > -ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 > +check ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 > check ovn-nbctl --wait=sb lr-nat-add lr0 dnat 192.168.2.5 10.0.0.5 > > ovn-sbctl dump-flows lr0 > sbflows > AT_CAPTURE_FILE([sbflows]) > > -OVS_WAIT_UNTIL([test 1 = $(grep lr_in_unsnat sbflows | \ > -grep "ip4 && ip4.dst == 192.168.2.1 && tcp && tcp.dst == 8080" -c) ]) > - > -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ > -grep "ip4 && ip4.dst == 192.168.2.4 && udp && udp.dst == 8080" -c) ]) > - > -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ > -grep "ip4 && ip4.dst == 192.168.2.5 && tcp && tcp.dst == 8080" -c) ]) > - > -AT_CHECK([test 0 = $(grep lr_in_unsnat sbflows | \ > -grep "ip4 && ip4.dst == 192.168.2.6 && tcp && tcp.dst == 8080" -c) ]) > +# There shoule be no flows for LB VIPs in lr_in_unsnat if the VIP is not a > +# dnat_and_snat or snat entry. > +AT_CHECK([grep "lr_in_unsnat" sbflows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.1 && tcp && tcp.dst == 8080), action=(next;) > + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.4 && udp && udp.dst == 8080), action=(next;) > + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.5 && tcp && tcp.dst == 8080), action=(next;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 192.168.2.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 192.168.2.4), action=(ct_snat;) > +]) > > AT_CLEANUP > ]) > @@ -1458,8 +1457,8 @@ ovn-nbctl set logical_router lr0 options:dnat_force_snat_ip=192.168.2.3 > ovn-nbctl --wait=sb sync > > AT_CHECK([ovn-sbctl lflow-list lr0 | grep lr_in_unsnat | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 192.168.2.3), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 192.168.2.3), action=(ct_snat;) > ]) > > AT_CLEANUP > @@ -3163,14 +3162,28 @@ ovn-sbctl dump-flows lr0 > lr0flows > AT_CAPTURE_FILE([lr0flows]) > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > ]) > > AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_dnat;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) > - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > ]) > > check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="20.0.0.4 aef0::4" > @@ -3180,23 +3193,37 @@ AT_CAPTURE_FILE([lr0flows]) > > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 20.0.0.4), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst == aef0::4), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 20.0.0.4), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst == aef0::4), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > ]) > > AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > ]) > > AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) > - table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) > - table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) > - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) > + table=2 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > ]) > > check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="router_ip" > @@ -3208,25 +3235,39 @@ AT_CHECK([grep "lr_in_ip_input" lr0flows | grep "priority=60" | sort], [0], [dnl > ]) > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > ]) > > AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > ]) > > AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) > - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > ]) > > check ovn-nbctl --wait=sb remove logical_router lr0 options chassis > @@ -3235,12 +3276,12 @@ ovn-sbctl dump-flows lr0 > lr0flows > AT_CAPTURE_FILE([lr0flows]) > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > ]) > > AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) > - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > ]) > > check ovn-nbctl set logical_router lr0 options:chassis=ch1 > @@ -3250,27 +3291,41 @@ ovn-sbctl dump-flows lr0 > lr0flows > AT_CAPTURE_FILE([lr0flows]) > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > ]) > > AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > ]) > > AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) > - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), action=(ct_snat(bef0::1);) > - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), action=(ct_snat(bef0::1);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > ]) > > check ovn-nbctl --wait=sb lb-add lb2 10.0.0.20:80 10.0.0.40:8080 > @@ -3280,20 +3335,35 @@ check ovn-nbctl --wait=sb lb-del lb1 > ovn-sbctl dump-flows lr0 > lr0flows > > AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.20 && tcp), action=(reg0 = 10.0.0.20; ct_dnat;) > ]) > > AT_CHECK([grep "lr_in_dnat" lr0flows | grep skip_snat_for_lb | sort], [0], [dnl > - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_dnat;) > - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_lb(backends=10.0.0.40:8080);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.20 && ct_label.natted == 1 && tcp), action=(flags.skip_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_lb(backends=10.0.0.40:8080);) > ]) > > AT_CHECK([grep "lr_out_snat" lr0flows | grep skip_snat_for_lb | sort], [0], [dnl > - table=1 (lr_out_snat ), priority=120 , match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > ]) > > AT_CLEANUP > @@ -3737,3 +3807,451 @@ AT_CHECK([ovn-trace --minimal 'inport == "sw1-port1" && eth.src == 50:54:00:00:0 > > AT_CLEANUP > ]) > + > +OVN_FOR_EACH_NORTHD([ > +AT_SETUP([ovn -- LR NAT flows]) > +ovn_start > + > +check ovn-nbctl \ > + -- ls-add sw0 \ > + -- lb-add lb0 10.0.0.10:80 10.0.0.4:8080 \ > + -- ls-lb-add sw0 lb0 > + > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > +check ovn-nbctl lsp-add sw0 sw0-lr0 > +check ovn-nbctl lsp-set-type sw0-lr0 router > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 > + > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > +]) > + > +# Create few dnat_and_snat entries > + > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.10 10.0.0.0/24 > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.20 10.0.0.3 > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.30 10.0.0.10 > + > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > +]) > + > +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 > + > +# Create a distributed gw port on lr0 > +check ovn-nbctl ls-add public > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 > + > +ovn-nbctl lsp-add public public-lr0 -- set Logical_Switch_Port public-lr0 \ > + type=router options:router-port=lr0-public \ > + -- lsp-set-addresses public-lr0 router > + > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) > +]) > + > +# Associate load balancer to lr0 > + > +check ovn-nbctl lb-add lb0 172.168.0.100:8082 "10.0.0.50:82,10.0.0.60:82" > + > +# No L4 > +check ovn-nbctl lb-add lb1 172.168.0.200 "10.0.0.80,10.0.0.81" > +check ovn-nbctl lb-add lb2 172.168.0.210:60 "10.0.0.50:6062, 10.0.0.60:6062" udp > + > +check ovn-nbctl lr-lb-add lr0 lb0 > +check ovn-nbctl lr-lb-add lr0 lb1 > +check ovn-nbctl lr-lb-add lr0 lb2 > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.80,10.0.0.81);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp && is_chassis_resident("cr-lr0-public")), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.4:8080 );) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:6062 ,10.0.0.60:6062);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.4 && tcp.src == 8080)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == 6062)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.80) || (ip4.src == 10.0.0.81)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) > +]) > + > +# Make the logical router as Gateway router > +check ovn-nbctl clear logical_router_port lr0-public gateway_chassis > +check ovn-nbctl set logical_router lr0 options:chassis=gw1 > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(next;) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(ct_lb(backends=10.0.0.80,10.0.0.81);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(ct_lb(backends= 10.0.0.50:82,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(ct_lb(backends= 10.0.0.50:6062,10.0.0.60:6062);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) > +]) > + > +# Set lb force snat logical router. > +check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="router_ip" > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 ,10.0.0.60:6062);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) > +]) > + > +# Add a LB VIP same as router ip. > +check ovn-nbctl lb-add lb0 172.168.0.10:9082 "10.0.0.50:82,10.0.0.60:82" > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 ,10.0.0.60:6062);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) > +]) > + > +# Add IPv6 router port and LB. > +check ovn-nbctl lrp-del lr0-sw0 > +check ovn-nbctl lrp-del lr0-public > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 aef0::1 > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 def0::10 > + > +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb1) > +ovn-nbctl set load_balancer $lb1_uuid vips:'"[[def0::2]]:8000"'='"@<:@aef0::2@:>@:80,@<:@aef0::3@:>@:80"' > + > +ovn-nbctl list load_Balancer > +check ovn-nbctl --wait=sb sync > + > +ovn-sbctl dump-flows lr0 > lr0flows > +AT_CAPTURE_FILE([lr0flows]) > + > +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl > + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip6.dst == def0::10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip6.dst == aef0::1), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) > + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) > +]) > + > +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl > + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) > + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip6.dst == def0::2 && tcp), action=(xxreg0 = def0::2; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl > + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) > + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && xxreg0 == def0::2 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 ,10.0.0.60:82);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 ,10.0.0.60:6062);) > + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && xxreg0 == def0::2 && tcp && tcp.dst == 8000), action=(flags.force_snat_for_lb = 1; ct_lb(backends=[[aef0::2]]:80,[[aef0::3]]:80);) > +]) > + > +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl > + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) > + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) > +]) > + > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl > + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) > + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) > +]) > + > +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl > + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-public"), action=(ct_snat(def0::10);) > + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw0"), action=(ct_snat(aef0::1);) > + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) > + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) > + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) > +]) > + > +AT_CLEANUP > +]) > diff --git a/tests/ovn.at b/tests/ovn.at > index bc494fcad9bb..ea1593197f21 100644 > --- a/tests/ovn.at > +++ b/tests/ovn.at > @@ -20571,7 +20571,7 @@ AT_CAPTURE_FILE([sbflows2]) > OVS_WAIT_FOR_OUTPUT( > [ovn-sbctl dump-flows > sbflows2 > ovn-sbctl dump-flows lr0 | grep ct_lb | grep priority=120 | sed 's/table=..//'], 0, > - [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80, 20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) > + [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80, 20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) > ]) > > # get the svc monitor mac. > @@ -20612,8 +20612,8 @@ AT_CHECK( > AT_CAPTURE_FILE([sbflows4]) > ovn-sbctl dump-flows lr0 > sbflows4 > AT_CHECK([grep lr_in_dnat sbflows4 | grep priority=120 | sed 's/table=..//' | sort], [0], [dnl > - (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) > - (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(drop;) > + (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) > + (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(drop;) > ]) > > # Delete sw0-p1 > diff --git a/tests/system-ovn.at b/tests/system-ovn.at > index 552fdae52665..4f104171bdba 100644 > --- a/tests/system-ovn.at > +++ b/tests/system-ovn.at > @@ -116,6 +116,7 @@ NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 | FORMAT_PING], \ > # Check conntrack entries. > AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.2) | \ > sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl > +icmp,orig=(src=172.16.1.2,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> > icmp,orig=(src=172.16.1.2,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> > ]) > > @@ -298,6 +299,7 @@ NS_CHECK_EXEC([alice1], [ping6 -q -c 3 -i 0.3 -w 2 fd30::2 | FORMAT_PING], \ > # Check conntrack entries. > AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd21::2) | \ > sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl > +icmpv6,orig=(src=fd21::2,dst=fd11::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> > icmpv6,orig=(src=fd21::2,dst=fd30::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> > ]) > > @@ -2197,11 +2199,12 @@ tcp,orig=(src=172.16.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(sr > ]) > > check_est_flows () { > - n=$(ovs-ofctl dump-flows br-int table=15 | grep \ > -"priority=120,ct_state=+est+trk,tcp,metadata=0x2,nw_dst=30.0.0.2,tp_dst=8000" \ > -| grep nat | sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') > + n=$(ovs-ofctl dump-flows br-int table=13 | grep \ > +"priority=100,tcp,metadata=0x2,nw_dst=30.0.0.2" | grep nat | > +sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') > > echo "n_packets=$n" > + test ! -z $n > test "$n" != 0 > } > > @@ -2222,7 +2225,7 @@ ovn-nbctl set load_balancer $uuid vips:'" 30.0.0.2:8000"'='"192.168.1.2:80,192.16 > > ovn-nbctl list load_balancer > ovn-sbctl dump-flows R2 > -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ > +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ > grep 'nat(src=20.0.0.2)']) > > dnl Test load-balancing that includes L4 ports in NAT. > @@ -2260,7 +2263,7 @@ ovn-nbctl set load_balancer $uuid vips:'" 30.0.0.2:8000"'='"192.168.1.2:80,192.16 > > ovn-nbctl list load_balancer > ovn-sbctl dump-flows R2 > -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ > +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ > grep 'nat(src=20.0.0.2)']) > > rm -f wget*.log > -- > 2.27.0 > > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
On 23/06/2021 03:00, Han Zhou wrote: > On Sat, Jun 19, 2021 at 2:52 AM Mark Gray <mark.d.gray@redhat.com> wrote: >> >> This patch addresses a number of interconnected issues with Gateway > Routers >> that have Load Balancing enabled: >> >> 1) In the router pipeline, we have the following stages to handle >> dnat and unsnat. >> >> - Stage 4 : lr_in_defrag (dnat zone) >> - Stage 5 : lr_in_unsnat (snat zone) >> - Stage 6 : lr_in_dnat (dnat zone) >> >> In the reply direction, the order of traversal of the tables >> "lr_in_defrag", "lr_in_unsnat" and "lr_in_dnat" adds incorrect >> datapath flows that check ct_state in the wrong conntrack zone. >> This is illustrated below where reply trafic enters the physical host >> port (6) and traverses DNAT zone (14), SNAT zone (default), back to the >> DNAT zone and then on to Logical Switch Port zone (22). The third >> flow is incorrectly checking the state from the SNAT zone instead >> of the DNAT zone. >> >> recirc_id(0),in_port(6),ct_state(-new-est-rel-rpl-trk) > actions:ct_clear,ct(zone=14),recirc(0xf) >> recirc_id(0xf),in_port(6) actions:ct(nat),recirc(0x10) >> recirc_id(0x10),in_port(6),ct_state(-new+est+trk) > actions:ct(zone=14,nat),recirc(0x11) >> recirc_id(0x11),in_port(6),ct_state(+new-est-rel-rpl+trk) actions: > ct(zone=22,nat),recirc(0x12) >> recirc_id(0x12),in_port(6),ct_state(-new+est-rel+rpl+trk) actions:5 >> >> Update the order of these tables to resolve this. >> >> 2) Efficiencies can be gained by using the ct_dnat action in the >> table "lr_in_defrag" instead of ct_next. This removes the need for the >> ct_dnat action for established Load Balancer flows avoiding a >> recirculation. >> >> 3) On a Gateway router with DNAT flows configured, the router will > translate >> the destination IP address from (A) to (B). Reply packets from (B) are >> correctly UNDNATed in the reverse direction. >> >> However, if a new connection is established from (B), this flow is never >> committed to conntrack and, as such, is never established. This will >> cause OVS datapath flows to be added that match on the ct.new flag. >> >> For software-only datapaths this is not a problem. However, for >> datapaths that offload these flows to hardware, this may be problematic >> as some devices are unable to offload flows that match on ct.new. >> >> This patch resolves this by committing these flows to the DNAT zone in >> the new "lr_out_post_undnat" stage. Although this could be done in the >> DNAT zone, by doing this in the new zone we can avoid a recirculation. >> >> Co-authored-by: Numan Siddique <numans@ovn.org> >> Signed-off-by: Mark Gray <mark.d.gray@redhat.com> >> Signed-off-by: Numan Siddique <numans@ovn.org> > > Thanks Mark and Numan. Please see some non-critical comments inlined. > >> --- >> northd/ovn-northd.8.xml | 143 +++++---- >> northd/ovn-northd.c | 111 +++++-- >> northd/ovn_northd.dl | 113 +++++-- >> tests/ovn-northd.at | 674 +++++++++++++++++++++++++++++++++++----- >> tests/ovn.at | 6 +- >> tests/system-ovn.at | 13 +- >> 6 files changed, 853 insertions(+), 207 deletions(-) >> >> diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml >> index 4074646029b4..d56a121d4d2e 100644 >> --- a/northd/ovn-northd.8.xml >> +++ b/northd/ovn-northd.8.xml >> @@ -2628,39 +2628,9 @@ icmp6 { >> </li> >> </ul> >> >> - <h3>Ingress Table 4: DEFRAG</h3> >> >> - <p> >> - This is to send packets to connection tracker for tracking and >> - defragmentation. It contains a priority-0 flow that simply moves > traffic >> - to the next table. >> - </p> >> - >> - <p> >> - If load balancing rules with virtual IP addresses (and ports) are >> - configured in <code>OVN_Northbound</code> database for a Gateway > router, >> - a priority-100 flow is added for each configured virtual IP address >> - <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip >> - && ip4.dst == <var>VIP</var></code>. For IPv6 > <var>VIPs</var>, >> - the flow matches <code>ip && ip6.dst == > <var>VIP</var></code>. >> - The flow uses the action <code>ct_next;</code> to send IP packets > to the >> - connection tracker for packet de-fragmentation and tracking before >> - sending it to the next table. >> - </p> >> - >> - <p> >> - If ECMP routes with symmetric reply are configured in the >> - <code>OVN_Northbound</code> database for a gateway router, a > priority-300 >> - flow is added for each router port on which symmetric replies are >> - configured. The matching logic for these ports essentially > reverses the >> - configured logic of the ECMP route. So for instance, a route with a >> - destination routing policy will instead match if the source IP > address >> - matches the static route's prefix. The flow uses the action >> - <code>ct_next</code> to send IP packets to the connection tracker > for >> - packet de-fragmentation and tracking before sending it to the next > table. >> - </p> >> >> - <h3>Ingress Table 5: UNSNAT</h3> >> + <h3>Ingress Table 4: UNSNAT</h3> >> >> <p> >> This is for already established connections' reverse traffic. >> @@ -2669,7 +2639,7 @@ icmp6 { >> unSNATted here. >> </p> >> >> - <p>Ingress Table 5: UNSNAT on Gateway and Distributed Routers</p> >> + <p>Ingress Table 4: UNSNAT on Gateway and Distributed Routers</p> >> <ul> >> <li> >> <p> >> @@ -2696,7 +2666,7 @@ icmp6 { >> </li> >> </ul> >> >> - <p>Ingress Table 5: UNSNAT on Gateway Routers</p> >> + <p>Ingress Table 4: UNSNAT on Gateway Routers</p> >> >> <ul> >> <li> >> @@ -2713,9 +2683,10 @@ icmp6 { >> <code>lb_force_snat_ip=router_ip</code> then for every logical > router >> port <var>P</var> attached to the Gateway router with the > router ip >> <var>B</var>, a priority-110 flow is added with the match >> - <code>inport == <var>P</var> && ip4.dst == > <var>B</var></code> or >> - <code>inport == <var>P</var> && ip6.dst == > <var>B</var></code> >> - with an action <code>ct_snat; </code>. >> + <code>inport == <var>P</var> && >> + ip4.dst == <var>B</var></code> or <code>inport == <var>P</var> >> + && ip6.dst == <var>B</var></code> with an action >> + <code>ct_snat; </code>. >> </p> >> >> <p> >> @@ -2745,7 +2716,7 @@ icmp6 { >> </li> >> </ul> >> >> - <p>Ingress Table 5: UNSNAT on Distributed Routers</p> >> + <p>Ingress Table 4: UNSNAT on Distributed Routers</p> >> >> <ul> >> <li> >> @@ -2776,6 +2747,40 @@ icmp6 { >> </li> >> </ul> >> >> + <h3>Ingress Table 5: DEFRAG</h3> >> + >> + <p> >> + This is to send packets to connection tracker for tracking and >> + defragmentation. It contains a priority-0 flow that simply moves > traffic >> + to the next table. >> + </p> >> + >> + <p> >> + If load balancing rules with virtual IP addresses (and ports) are >> + configured in <code>OVN_Northbound</code> database for a Gateway > router, >> + a priority-100 flow is added for each configured virtual IP address >> + <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip >> + && ip4.dst == <var>VIP</var></code>. For IPv6 > <var>VIPs</var>, >> + the flow matches <code>ip && ip6.dst == > <var>VIP</var></code>. >> + The flow applies the action <code>reg0 = <var>VIP</var> > > nit: for IPv6 it is xxreg0. > >> + && ct_dnat;</code> to send IP packets to the >> + connection tracker for packet de-fragmentation and to dnat the >> + destination IP for the committed connection before sending it to > the >> + next table. >> + </p> >> + >> + <p> >> + If ECMP routes with symmetric reply are configured in the >> + <code>OVN_Northbound</code> database for a gateway router, a > priority-300 >> + flow is added for each router port on which symmetric replies are >> + configured. The matching logic for these ports essentially > reverses the >> + configured logic of the ECMP route. So for instance, a route with a >> + destination routing policy will instead match if the source IP > address >> + matches the static route's prefix. The flow uses the action >> + <code>ct_next</code> to send IP packets to the connection tracker > for >> + packet de-fragmentation and tracking before sending it to the next > table. >> + </p> >> + >> <h3>Ingress Table 6: DNAT</h3> >> >> <p> >> @@ -2828,19 +2833,28 @@ icmp6 { >> </li> >> >> <li> >> - For all the configured load balancing rules for a router in >> - <code>OVN_Northbound</code> database that includes a L4 port >> - <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 address >> - <var>VIP</var>, a priority-120 flow that matches on >> - <code>ct.est && ip && ip4.dst == <var>VIP</var> >> - && <var>P</var> && <var>P</var>.dst == <var>PORT >> - </var></code> (<code>ip6.dst == <var>VIP</var></code> in the > IPv6 case) >> - with an action of <code>ct_dnat;</code>. If the router is >> - configured to force SNAT any load-balanced packets, the above > action >> - will be replaced by <code>flags.force_snat_for_lb = 1; > ct_dnat;</code>. >> - If the load balancing rule is configured with > <code>skip_snat</code> >> - set to true, the above action will be replaced by >> - <code>flags.skip_snat_for_lb = 1; ct_dnat;</code>. >> + <p> >> + For all the configured load balancing rules for a router in >> + <code>OVN_Northbound</code> database that includes a L4 port >> + <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 > address >> + <var>VIP</var>, a priority-120 flow that matches on >> + <code>ct.est && ip && reg0 == <var>VIP</var> >> + && <var>P</var> && <var>P</var>.dst == > <var>PORT >> + </var></code> (<code>xxreg0 == <var>VIP</var></code> in the >> + IPv6 case) with an action of <code>next;</code>. If the router > is >> + configured to force SNAT any load-balanced packets, the above > action >> + will be replaced by <code>flags.force_snat_for_lb = 1; > next;</code>. >> + If the load balancing rule is configured with > <code>skip_snat</code> >> + set to true, the above action will be replaced by >> + <code>flags.skip_snat_for_lb = 1; next;</code>. >> + </p> >> + >> + <p> >> + Previous table <code>lr_in_defrag</code> sets the register >> + <code>reg0</code> (or <code>xxreg0</code> for IPv6) and does >> + <code>ct_dnat</code>. Hence for established traffic, this >> + table just advances the packet to the next stage. >> + </p> >> </li> >> > In this section "Ingress Table 6: DNAT", there are several paragraphs with > very similar text but only minor differences on the match condition, such > as "includes just an IP address ...", and you need to update for all those > paragraphs. I understand it is painful to update the redundant information, > and it may be also painful for the readers, but it seems even worse if it > is incorrect. Not sure if there is a better way to maintain this document. > (the DDlog code is easier to understand than the document) Sorry, this appears to be an unintentional omission. I will update this section. It would be good if we found an easier way to keep this up-to-date, and as you state, keep it correct. It is a key resource for debugging. Unfortunately the code isn't particularly self-documenting :(. I guess its something to have a think about for the future .. > >> <li> >> @@ -3876,7 +3890,26 @@ nd_ns { >> </li> >> </ul> >> >> - <h3>Egress Table 1: SNAT</h3> >> + <h3>Egress Table 1: Post UNDNAT on Gateway Routers</h3> >> + >> + <p> >> + <ul> >> + <li> >> + A priority-50 logical flow is added that commits any untracked > flows >> + from the previous table <code>lr_out_undnat</code>. This flow >> + matches on <code>ct.new && ip</code> with action >> + <code>ct_commit { } ; next; </code>. >> + </li> >> + >> + <li> >> + A priority-0 logical flow with match <code>1</code> has actions >> + <code>next;</code>. >> + </li> >> + >> + </ul> >> + </p> >> + >> + <h3>Egress Table 2: SNAT</h3> >> >> <p> >> Packets that are configured to be SNATed get their source IP > address >> @@ -3892,7 +3925,7 @@ nd_ns { >> </li> >> </ul> >> >> - <p>Egress Table 1: SNAT on Gateway Routers</p> >> + <p>Egress Table 2: SNAT on Gateway Routers</p> >> >> <ul> >> <li> >> @@ -3991,7 +4024,7 @@ nd_ns { >> </li> >> </ul> >> >> - <p>Egress Table 1: SNAT on Distributed Routers</p> >> + <p>Egress Table 2: SNAT on Distributed Routers</p> >> >> <ul> >> <li> >> @@ -4051,7 +4084,7 @@ nd_ns { >> </li> >> </ul> >> >> - <h3>Egress Table 2: Egress Loopback</h3> >> + <h3>Egress Table 3: Egress Loopback</h3> >> >> <p> >> For distributed logical routers where one of the logical router >> @@ -4120,7 +4153,7 @@ clone { >> </li> >> </ul> >> >> - <h3>Egress Table 3: Delivery</h3> >> + <h3>Egress Table 4: Delivery</h3> >> >> <p> >> Packets that reach this table are ready for delivery. It contains: >> diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c >> index d97ab4a5b39c..27e5fbea9f4f 100644 >> --- a/northd/ovn-northd.c >> +++ b/northd/ovn-northd.c >> @@ -187,8 +187,8 @@ enum ovn_stage { >> PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, > "lr_in_lookup_neighbor") \ >> PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, > "lr_in_learn_neighbor") \ >> PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") > \ >> - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 4, "lr_in_defrag") > \ >> - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 5, "lr_in_unsnat") > \ >> + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") > \ >> + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") > \ >> PIPELINE_STAGE(ROUTER, IN, DNAT, 6, "lr_in_dnat") > \ >> PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, > "lr_in_ecmp_stateful") \ >> PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, > "lr_in_nd_ra_options") \ >> @@ -204,10 +204,11 @@ enum ovn_stage { >> PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 18, > "lr_in_arp_request") \ >> \ >> /* Logical router egress stages. */ \ >> - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ >> - PIPELINE_STAGE(ROUTER, OUT, SNAT, 1, "lr_out_snat") \ >> - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 2, "lr_out_egr_loop") \ >> - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 3, "lr_out_delivery") >> + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ >> + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 1, "lr_out_post_undnat") \ >> + PIPELINE_STAGE(ROUTER, OUT, SNAT, 2, "lr_out_snat") \ >> + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 3, "lr_out_egr_loop") \ >> + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 4, "lr_out_delivery") >> >> #define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ >> S_##DP_TYPE##_##PIPELINE##_##STAGE \ >> @@ -643,6 +644,12 @@ struct ovn_datapath { >> /* Multicast data. */ >> struct mcast_info mcast_info; >> >> + /* Applies to only logical router datapath. >> + * True if logical router is a gateway router. i.e options:chassis > is set. >> + * If this is true, then 'l3dgw_port' and 'l3redirect_port' will be >> + * ignored. */ >> + bool is_gw_router; >> + > > This looks redundant with the field l3dgw_port. The newly added bool is > used only once, and the other places are still using !l3dgw_port to check > if it is GW router. I suggest either keep using !l3dgw_port or unifying all > the checks to use the new bool. Numans updated this to use 'is_gw_router' everywhere > >> /* OVN northd only needs to know about the logical router gateway > port for >> * NAT on a distributed router. This "distributed gateway port" is >> * populated only when there is a gateway chassis specified for one > of >> @@ -1247,6 +1254,9 @@ join_datapaths(struct northd_context *ctx, struct > hmap *datapaths, >> init_mcast_info_for_datapath(od); >> init_nat_entries(od); >> init_lb_ips(od); >> + if (smap_get(&od->nbr->options, "chassis")) { >> + od->is_gw_router = true; >> + } >> ovs_list_push_back(lr_list, &od->lr_list); >> } >> } >> @@ -8731,20 +8741,33 @@ add_router_lb_flow(struct hmap *lflows, struct > ovn_datapath *od, >> } >> >> /* A match and actions for established connections. */ >> - char *est_match = xasprintf("ct.est && %s", ds_cstr(match)); >> + struct ds est_match = DS_EMPTY_INITIALIZER; >> + ds_put_format(&est_match, >> + "ct.est && ip && %sreg0 == %s && ct_label.natted == 1", >> + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) ? "" : "xx", >> + lb_vip->vip_str); >> + if (lb_vip->vip_port) { >> + ds_put_format(&est_match, " && %s", proto); >> + } >> + if (od->l3redirect_port && >> + (lb_vip->n_backends || !lb_vip->empty_backend_rej)) { >> + ds_put_format(&est_match, " && is_chassis_resident(%s)", >> + od->l3redirect_port->json_key); >> + } > > This part is a little redundant with constructing the "match" in the caller > of this function. The only difference is matching ip.dst or "reg0/xxreg0". > It is not anything critical but it may be better to put the logic at the > same place, maybe just move the logic from the caller to this function. I have refactored this as you suggested, I think it looks marginally cleaner. Although this whole section of the code could do with a larger refactor IMO. A task for another day ... > >> if (snat_type == FORCE_SNAT || snat_type == SKIP_SNAT) { >> - char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; > ct_dnat;", >> + char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; next;", >> snat_type == SKIP_SNAT ? "skip" : "force"); >> ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, >> - est_match, est_actions, &lb->header_); >> + ds_cstr(&est_match), est_actions, >> + &lb->header_); >> free(est_actions); >> } else { >> ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, >> - est_match, "ct_dnat;", &lb->header_); >> + ds_cstr(&est_match), "next;", > &lb->header_); >> } >> >> free(new_match); >> - free(est_match); >> + ds_destroy(&est_match); >> >> const char *ip_match = NULL; >> if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { >> @@ -8829,8 +8852,8 @@ add_router_lb_flow(struct hmap *lflows, struct > ovn_datapath *od, >> static void >> build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, >> struct hmap *lbs, struct shash *meter_groups, >> - struct sset *nat_entries, struct ds *match, >> - struct ds *actions) >> + struct sset *nat_entries, >> + struct ds *match, struct ds *actions) >> { >> /* A set to hold all ips that need defragmentation and tracking. */ >> struct sset all_ips = SSET_INITIALIZER(&all_ips); >> @@ -8852,10 +8875,17 @@ build_lrouter_lb_flows(struct hmap *lflows, > struct ovn_datapath *od, >> for (size_t j = 0; j < lb->n_vips; j++) { >> struct ovn_lb_vip *lb_vip = &lb->vips[j]; >> struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[j]; >> + >> + bool is_udp = nullable_string_is_equal(nb_lb->protocol, > "udp"); >> + bool is_sctp = nullable_string_is_equal(nb_lb->protocol, >> + "sctp"); >> + const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : > "tcp"; >> + >> ds_clear(actions); >> build_lb_vip_actions(lb_vip, lb_vip_nb, actions, >> lb->selection_fields, false); >> >> + struct ds defrag_actions = DS_EMPTY_INITIALIZER; >> if (!sset_contains(&all_ips, lb_vip->vip_str)) { >> sset_add(&all_ips, lb_vip->vip_str); >> /* If there are any load balancing rules, we should send >> @@ -8867,17 +8897,28 @@ build_lrouter_lb_flows(struct hmap *lflows, > struct ovn_datapath *od, >> * 2. If there are L4 ports in load balancing rules, we >> * need the defragmentation to match on L4 ports. */ >> ds_clear(match); >> + ds_clear(&defrag_actions); >> if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { >> ds_put_format(match, "ip && ip4.dst == %s", >> lb_vip->vip_str); >> + ds_put_format(&defrag_actions, "reg0 = %s; ct_dnat;", >> + lb_vip->vip_str); >> } else { >> ds_put_format(match, "ip && ip6.dst == %s", >> lb_vip->vip_str); >> + ds_put_format(&defrag_actions, "xxreg0 = %s; > ct_dnat;", >> + lb_vip->vip_str); >> + } >> + >> + if (lb_vip->vip_port) { >> + ds_put_format(match, " && %s", proto); >> } >> ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, >> - 100, ds_cstr(match), "ct_next;", >> + 100, ds_cstr(match), >> + ds_cstr(&defrag_actions), >> &nb_lb->header_); >> } >> + ds_destroy(&defrag_actions); >> >> /* Higher priority rules are added for load-balancing in DNAT >> * table. For every match (on a VIP[:port]), we add two > flows >> @@ -8886,18 +8927,14 @@ build_lrouter_lb_flows(struct hmap *lflows, > struct ovn_datapath *od, >> * flow is for ct.est with an action of "ct_dnat;". */ >> ds_clear(match); >> if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { >> - ds_put_format(match, "ip && ip4.dst == %s", >> + ds_put_format(match, "ip && reg0 == %s", >> lb_vip->vip_str); >> } else { >> - ds_put_format(match, "ip && ip6.dst == %s", >> + ds_put_format(match, "ip && xxreg0 == %s", >> lb_vip->vip_str); >> } >> >> int prio = 110; >> - bool is_udp = nullable_string_is_equal(nb_lb->protocol, > "udp"); >> - bool is_sctp = nullable_string_is_equal(nb_lb->protocol, >> - "sctp"); >> - const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : > "tcp"; >> >> if (lb_vip->vip_port) { >> ds_put_format(match, " && %s && %s.dst == %d", proto, >> @@ -11400,8 +11437,7 @@ build_lrouter_out_undnat_flow(struct hmap > *lflows, struct ovn_datapath *od, >> * part of a reply. We undo the DNAT here. >> * >> * Note that this only applies for NAT on a distributed router. >> - * Undo DNAT on a gateway router is done in the ingress DNAT >> - * pipeline stage. */ >> + */ >> if (!od->l3dgw_port || >> (strcmp(nat->type, "dnat") && strcmp(nat->type, > "dnat_and_snat"))) { >> return; >> @@ -11681,9 +11717,22 @@ build_lrouter_nat_defrag_and_lb(struct > ovn_datapath *od, >> ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); >> ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); >> ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); >> + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); >> ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); >> ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", > "next;"); >> >> + /* For Gateway routers, if the gateway router has load balancer or > DNAT >> + * rules, we commit newly initiated connections in the reply > direction >> + * to the DNAT zone. This ensures that these flows are tracked. If > the flow >> + * was not committed, it would produce ongoing datapath flows with > the >> + * ct.new flag set. Some NICs are unable to offload these flows. >> + */ >> + if (od->is_gw_router && >> + (od->nbr->n_nat || od->nbr->n_load_balancer)) { >> + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, >> + "ip && ct.new", "ct_commit { } ; next; "); >> + } >> + > > Why do the commit for GW routers ONLY? I think it may be better to keep the > behavior consistent for both distributed routers and GW routers. I > understand that distributed routers has flows that match each individual > backends IP [&& port], so there are less chances to go through this stage, > but it is still possible if a backend initiates a connection to external > (when L4 port is not specified for the VIP, or the source port happens to > be the VIP's port). And it seems not harmful to add the flow for > distributed routers if only very few connections would hit this flow. > Ok - makes sense. Updated > Moreover, I think it is better to keep the behavior consistent between the > two types of routers also regarding the individual backends IP [&& port] > check. There were control plane scale concerns discussed. However, from > what I observed with DP group feature enabled, the impact should be > minimal. So I'd suggest introducing the same checks for GW router datapaths > before going through the UNDNAT stage, to avoid the extra recirculation for > most use cases. It would be great to have a knob to turn it off when there > is scale concerns. However, please feel free to add it as a follow-up patch > if it is ok. > >> /* Send the IPv6 NS packets to next table. When ovn-controller >> * generates IPv6 NS (for the action - nd_ns{}), the injected >> * packet would go through conntrack - which is not required. */ >> @@ -11848,18 +11897,12 @@ build_lrouter_nat_defrag_and_lb(struct > ovn_datapath *od, >> od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); >> } >> } >> - >> - /* For gateway router, re-circulate every packet through >> - * the DNAT zone. This helps with the following. >> - * >> - * Any packet that needs to be unDNATed in the reverse >> - * direction gets unDNATed. Ideally this could be done in >> - * the egress pipeline. But since the gateway router >> - * does not have any feature that depends on the source >> - * ip address being external IP address for IP routing, >> - * we can do it here, saving a future re-circulation. */ >> - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, >> - "ip", "flags.loopback = 1; ct_dnat;"); >> + /* For gateway router, re-circulate every packet through the > DNAT zone >> + * so that packets that need to be unDNATed in the reverse > direction >> + * get unDNATed. >> + */ >> + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, >> + "ip", "flags.loopback = 1; ct_dnat;"); > > This change looks reasonable to me, because I don't understand the comment > above regarding how doing the UNDNAT in the ROUTER_IN_DNAT stage would save > a recirculation. > Could you update this part in the ovn-northd.xml document to reflect this > change as well? It is still mentioned in the document "For Gateway routers, > the unDNAT processing is carried out in the ingress DNAT table." Done > > Thanks, > Han > >> } >> >> /* Load balancing and packet defrag are only valid on >> diff --git a/northd/ovn_northd.dl b/northd/ovn_northd.dl >> index 3afa80a3b549..727641ac6ae1 100644 >> --- a/northd/ovn_northd.dl >> +++ b/northd/ovn_northd.dl >> @@ -1454,8 +1454,8 @@ function s_ROUTER_IN_ADMISSION(): Stage { > Stage{Ingress, 0, "lr_in_admiss >> function s_ROUTER_IN_LOOKUP_NEIGHBOR(): Stage { Stage{Ingress, 1, > "lr_in_lookup_neighbor"} } >> function s_ROUTER_IN_LEARN_NEIGHBOR(): Stage { Stage{Ingress, 2, > "lr_in_learn_neighbor"} } >> function s_ROUTER_IN_IP_INPUT(): Stage { Stage{Ingress, 3, > "lr_in_ip_input"} } >> -function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 4, > "lr_in_defrag"} } >> -function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 5, > "lr_in_unsnat"} } >> +function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 4, > "lr_in_unsnat"} } >> +function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 5, > "lr_in_defrag"} } >> function s_ROUTER_IN_DNAT(): Stage { Stage{Ingress, 6, > "lr_in_dnat"} } >> function s_ROUTER_IN_ECMP_STATEFUL(): Stage { Stage{Ingress, 7, > "lr_in_ecmp_stateful"} } >> function s_ROUTER_IN_ND_RA_OPTIONS(): Stage { Stage{Ingress, 8, > "lr_in_nd_ra_options"} } >> @@ -1472,9 +1472,10 @@ function s_ROUTER_IN_ARP_REQUEST(): Stage { > Stage{Ingress, 18, "lr_in_arp_re >> >> /* Logical router egress stages. */ >> function s_ROUTER_OUT_UNDNAT(): Stage { Stage{ Egress, 0, > "lr_out_undnat"} } >> -function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 1, > "lr_out_snat"} } >> -function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 2, > "lr_out_egr_loop"} } >> -function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 3, > "lr_out_delivery"} } >> +function s_ROUTER_OUT_POST_UNDNAT(): Stage { Stage{ Egress, 1, > "lr_out_post_undnat"} } >> +function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 2, > "lr_out_snat"} } >> +function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 3, > "lr_out_egr_loop"} } >> +function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 4, > "lr_out_delivery"} } >> >> /* >> * OVS register usage: >> @@ -2886,7 +2887,8 @@ for (&Switch(._uuid = ls_uuid)) { >> function get_match_for_lb_key(ip_address: v46_ip, >> port: bit<16>, >> protocol: Option<string>, >> - redundancy: bool): string = { >> + redundancy: bool, >> + use_nexthop_reg: bool): string = { >> var port_match = if (port != 0) { >> var proto = if (protocol == Some{"udp"}) { >> "udp" >> @@ -2900,8 +2902,18 @@ function get_match_for_lb_key(ip_address: v46_ip, >> }; >> >> var ip_match = match (ip_address) { >> - IPv4{ipv4} -> "ip4.dst == ${ipv4}", >> - IPv6{ipv6} -> "ip6.dst == ${ipv6}" >> + IPv4{ipv4} -> >> + if (use_nexthop_reg) { >> + "${rEG_NEXT_HOP()} == ${ipv4}" >> + } else { >> + "ip4.dst == ${ipv4}" >> + }, >> + IPv6{ipv6} -> >> + if (use_nexthop_reg) { >> + "xx${rEG_NEXT_HOP()} == ${ipv6}" >> + } else { >> + "ip6.dst == ${ipv6}" >> + } >> }; >> >> if (redundancy) { "ip && " } else { "" } ++ ip_match ++ port_match >> @@ -2935,7 +2947,11 @@ function build_lb_vip_actions(lbvip: > Intern<LBVIPWithStatus>, >> for (pair in lbvip.backends) { >> (var backend, var up) = pair; >> if (up) { >> - > up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") >> + if (backend.port != 0) { >> + > up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") >> + } else { >> + up_backends.insert("${backend.ip.to_bracketed_string()}") >> + } >> } >> }; >> >> @@ -2981,7 +2997,7 @@ Flow(.logical_datapath = sw._uuid, >> >> build_lb_vip_actions(lbvip, s_SWITCH_OUT_QOS_MARK(), actions0 ++ > actions1) >> }, >> - var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, > lbvip.vip_port, lb.protocol, false). >> + var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, > lbvip.vip_port, lb.protocol, false, false). >> >> /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). >> * Packets that don't need hairpinning should continue processing. >> @@ -3019,7 +3035,7 @@ for (&Switch(._uuid = ls_uuid, .has_lb_vip = true)) > { >> .__match = "ip && ct.new && ct.trk && ${rEGBIT_HAIRPIN()} == 1", >> .actions = "ct_snat_to_vip; next;", >> .external_ids = stage_hint(ls_uuid)); >> - >> + >> /* If packet needs to be hairpinned, for established sessions there >> * should already be an SNAT conntrack entry. >> */ >> @@ -5379,13 +5395,14 @@ function default_allow_flow(datapath: uuid, > stage: Stage): Flow { >> .actions = "next;", >> .external_ids = map_empty()} >> } >> -for (&Router(._uuid = lr_uuid)) { >> +for (r in &Router(._uuid = lr_uuid)) { >> /* Packets are allowed by default. */ >> Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DEFRAG())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_UNSNAT())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_SNAT())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DNAT())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_UNDNAT())]; >> + Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_POST_UNDNAT())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_EGR_LOOP())]; >> Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_ECMP_STATEFUL())]; >> >> @@ -5400,6 +5417,25 @@ for (&Router(._uuid = lr_uuid)) { >> .external_ids = map_empty()) >> } >> >> +for (r in &Router(._uuid = lr_uuid, >> + .is_gateway = is_gateway, >> + .nat = nat, >> + .load_balancer = load_balancer) >> + if is_gateway and (not is_empty(nat) or not > is_empty(load_balancer))) { >> + /* For Gateway routers, if the gateway router has load balancer or > DNAT >> + * rules, we commit newly initiated connections in the reply > direction >> + * to the DNAT zone. This ensures that these flows are tracked. If > the flow >> + * was not committed, it would produce ongoing datapath flows with > the >> + * ct.new flag set. Some NICs are unable to offload these flows. >> + */ >> + Flow(.logical_datapath = lr_uuid, >> + .stage = s_ROUTER_OUT_POST_UNDNAT(), >> + .priority = 50, >> + .__match = "ip && ct.new", >> + .actions = "ct_commit { } ; next; ", >> + .external_ids = map_empty()) >> +} >> + >> Flow(.logical_datapath = lr, >> .stage = s_ROUTER_OUT_SNAT(), >> .priority = 120, >> @@ -5438,7 +5474,7 @@ function lrouter_nat_add_ext_ip_match( >> Some{AllowedExtIps{__as}} -> (" && ${ipX}.${dir} == $${__as.name}", > None), >> Some{ExemptedExtIps{__as}} -> { >> /* Priority of logical flows corresponding to > exempted_ext_ips is >> - * +1 of the corresponding regulr NAT rule. >> + * +1 of the corresponding regular NAT rule. >> * For example, if we have following NAT rule and we > associate >> * exempted external ips to it: >> * "ovn-nbctl lr-nat-add router dnat_and_snat 10.15.24.139 > 50.0.0.11" >> @@ -5746,8 +5782,7 @@ for (r in &Router(._uuid = lr_uuid, >> * part of a reply. We undo the DNAT here. >> * >> * Note that this only applies for NAT on a distributed > router. >> - * Undo DNAT on a gateway router is done in the ingress DNAT >> - * pipeline stage. */ >> + */ >> if ((nat.nat.__type == "dnat" or nat.nat.__type == > "dnat_and_snat")) { >> Some{var gwport} = l3dgw_port in >> var __match = >> @@ -5953,16 +5988,11 @@ for (r in &Router(._uuid = lr_uuid, >> .context = "lb"); >> >> /* For gateway router, re-circulate every packet through >> - * the DNAT zone. This helps with the following. >> - * >> - * Any packet that needs to be unDNATed in the reverse >> - * direction gets unDNATed. Ideally this could be done in >> - * the egress pipeline. But since the gateway router >> - * does not have any feature that depends on the source >> - * ip address being external IP address for IP routing, >> - * we can do it here, saving a future re-circulation. */ >> + * the DNAT zone so that packets that need to be unDNATed in the > reverse >> + * direction get unDNATed. >> + */ >> Flow(.logical_datapath = lr_uuid, >> - .stage = s_ROUTER_IN_DNAT(), >> + .stage = s_ROUTER_OUT_UNDNAT(), >> .priority = 50, >> .__match = "ip", >> .actions = "flags.loopback = 1; ct_dnat;", >> @@ -6024,7 +6054,16 @@ for (RouterLBVIP( >> * pick a DNAT ip address from a group. >> * 2. If there are L4 ports in load balancing rules, we >> * need the defragmentation to match on L4 ports. */ >> - var __match = "ip && ${ipX}.dst == ${ip_address}" in >> + var match1 = "ip && ${ipX}.dst == ${ip_address}" in >> + var match2 = >> + if (port != 0) { >> + " && ${proto}" >> + } else { >> + "" >> + } in >> + var __match = match1 ++ match2 in >> + var xx = ip_address.xxreg() in >> + var __actions = "${xx}${rEG_NEXT_HOP()} = ${ip_address}; > ct_dnat;" in >> /* One of these flows must be created for each unique LB VIP > address. >> * We create one for each VIP:port pair; flows with the same IP > and >> * different port numbers will produce identical flows that will >> @@ -6033,7 +6072,7 @@ for (RouterLBVIP( >> .stage = s_ROUTER_IN_DEFRAG(), >> .priority = 100, >> .__match = __match, >> - .actions = "ct_next;", >> + .actions = __actions, >> .external_ids = stage_hint(lb._uuid)); >> >> /* Higher priority rules are added for load-balancing in DNAT >> @@ -6041,7 +6080,8 @@ for (RouterLBVIP( >> * via add_router_lb_flow(). One flow is for specific matching >> * on ct.new with an action of "ct_lb($targets);". The other >> * flow is for ct.est with an action of "ct_dnat;". */ >> - var match1 = "ip && ${ipX}.dst == ${ip_address}" in >> + var xx = ip_address.xxreg() in >> + var match1 = "ip && ${xx}${rEG_NEXT_HOP()} == ${ip_address}" in >> (var prio, var match2) = >> if (port != 0) { >> (120, " && ${proto} && ${proto}.dst == ${port}") >> @@ -6056,12 +6096,21 @@ for (RouterLBVIP( >> var snat_for_lb = snat_for_lb(r.options, lb) in >> { >> /* A match and actions for established connections. */ >> - var est_match = "ct.est && " ++ __match in >> + var est_match = "ct.est && " ++ match1 ++ " && > ct_label.natted == 1" ++ >> + if (port != 0) { >> + " && ${proto}" >> + } else { >> + "" >> + } ++ >> + match ((l3dgw_port, backends != "" or > lb.options.get_bool_def("reject", false))) { >> + (Some{gwport}, true) -> " && > is_chassis_resident(${redirect_port_name})", >> + _ -> "" >> + } in >> var actions = >> match (snat_for_lb) { >> - SkipSNAT -> "flags.skip_snat_for_lb = 1; ct_dnat;", >> - ForceSNAT -> "flags.force_snat_for_lb = 1; ct_dnat;", >> - _ -> "ct_dnat;" >> + SkipSNAT -> "flags.skip_snat_for_lb = 1; next;", >> + ForceSNAT -> "flags.force_snat_for_lb = 1; next;", >> + _ -> "next;" >> } in >> Flow(.logical_datapath = lr_uuid, >> .stage = s_ROUTER_IN_DNAT(), >> @@ -6152,7 +6201,7 @@ Flow(.logical_datapath = r._uuid, >> r.load_balancer.contains(lb._uuid), >> var __match >> = "ct.new && " ++ >> - get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, > lb.protocol, true) ++ >> + get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, > lb.protocol, true, true) ++ >> match (r.l3dgw_port) { >> Some{gwport} -> " && > is_chassis_resident(${r.redirect_port_name})", >> _ -> "" >> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at >> index d81975cb18a6..accaa87033f4 100644 >> --- a/tests/ovn-northd.at >> +++ b/tests/ovn-northd.at >> @@ -1406,40 +1406,39 @@ AT_SETUP([ovn -- Load balancer VIP in NAT > entries]) >> AT_SKIP_IF([test $HAVE_PYTHON = no]) >> ovn_start >> >> -ovn-nbctl lr-add lr0 >> -ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 >> -ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 >> +check ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 >> >> -ovn-nbctl set logical_router lr0 options:chassis=ch1 >> +check ovn-nbctl set logical_router lr0 options:chassis=ch1 >> >> -ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" >> -ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp >> -ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" >> -ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" >> +check ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" >> +check ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp >> +check ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" >> +check ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" >> >> -ovn-nbctl lr-lb-add lr0 lb1 >> -ovn-nbctl lr-lb-add lr0 lb2 >> -ovn-nbctl lr-lb-add lr0 lb3 >> -ovn-nbctl lr-lb-add lr0 lb4 >> +check ovn-nbctl lr-lb-add lr0 lb1 >> +check ovn-nbctl lr-lb-add lr0 lb2 >> +check ovn-nbctl lr-lb-add lr0 lb3 >> +check ovn-nbctl lr-lb-add lr0 lb4 >> >> -ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 >> -ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 >> +check ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 >> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 >> check ovn-nbctl --wait=sb lr-nat-add lr0 dnat 192.168.2.5 10.0.0.5 >> >> ovn-sbctl dump-flows lr0 > sbflows >> AT_CAPTURE_FILE([sbflows]) >> >> -OVS_WAIT_UNTIL([test 1 = $(grep lr_in_unsnat sbflows | \ >> -grep "ip4 && ip4.dst == 192.168.2.1 && tcp && tcp.dst == 8080" -c) ]) >> - >> -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ >> -grep "ip4 && ip4.dst == 192.168.2.4 && udp && udp.dst == 8080" -c) ]) >> - >> -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ >> -grep "ip4 && ip4.dst == 192.168.2.5 && tcp && tcp.dst == 8080" -c) ]) >> - >> -AT_CHECK([test 0 = $(grep lr_in_unsnat sbflows | \ >> -grep "ip4 && ip4.dst == 192.168.2.6 && tcp && tcp.dst == 8080" -c) ]) >> +# There shoule be no flows for LB VIPs in lr_in_unsnat if the VIP is not > a >> +# dnat_and_snat or snat entry. >> +AT_CHECK([grep "lr_in_unsnat" sbflows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst > == 192.168.2.1 && tcp && tcp.dst == 8080), action=(next;) >> + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst > == 192.168.2.4 && udp && udp.dst == 8080), action=(next;) >> + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst > == 192.168.2.5 && tcp && tcp.dst == 8080), action=(next;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 192.168.2.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 192.168.2.4), action=(ct_snat;) >> +]) >> >> AT_CLEANUP >> ]) >> @@ -1458,8 +1457,8 @@ ovn-nbctl set logical_router lr0 > options:dnat_force_snat_ip=192.168.2.3 >> ovn-nbctl --wait=sb sync >> >> AT_CHECK([ovn-sbctl lflow-list lr0 | grep lr_in_unsnat | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst > == 192.168.2.3), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst > == 192.168.2.3), action=(ct_snat;) >> ]) >> >> AT_CLEANUP >> @@ -3163,14 +3162,28 @@ ovn-sbctl dump-flows lr0 > lr0flows >> AT_CAPTURE_FILE([lr0flows]) >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> ]) >> >> AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_dnat;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(ct_lb(backends=10.0.0.4:8080);) >> - table=6 (lr_in_dnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), > action=(ct_lb(backends=10.0.0.4:8080);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> ]) >> >> check ovn-nbctl --wait=sb set logical_router lr0 > options:lb_force_snat_ip="20.0.0.4 aef0::4" >> @@ -3180,23 +3193,37 @@ AT_CAPTURE_FILE([lr0flows]) >> >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst > == 20.0.0.4), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst > == aef0::4), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst > == 20.0.0.4), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst > == aef0::4), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> ]) >> >> AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_dnat;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) >> - table=6 (lr_in_dnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> ]) >> >> AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> - table=1 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> - table=1 (lr_out_snat ), priority=100 , > match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) >> - table=1 (lr_out_snat ), priority=100 , > match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) >> - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=100 , > match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) >> + table=2 (lr_out_snat ), priority=100 , > match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> ]) >> >> check ovn-nbctl --wait=sb set logical_router lr0 > options:lb_force_snat_ip="router_ip" >> @@ -3208,25 +3235,39 @@ AT_CHECK([grep "lr_in_ip_input" lr0flows | grep > "priority=60" | sort], [0], [dnl >> ]) >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> ]) >> >> AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_dnat;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) >> - table=6 (lr_in_dnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> ]) >> >> AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> - table=1 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.100);) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), > action=(ct_snat(20.0.0.1);) >> - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.100);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), > action=(ct_snat(20.0.0.1);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> ]) >> >> check ovn-nbctl --wait=sb remove logical_router lr0 options chassis >> @@ -3235,12 +3276,12 @@ ovn-sbctl dump-flows lr0 > lr0flows >> AT_CAPTURE_FILE([lr0flows]) >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> ]) >> >> AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> - table=1 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> ]) >> >> check ovn-nbctl set logical_router lr0 options:chassis=ch1 >> @@ -3250,27 +3291,41 @@ ovn-sbctl dump-flows lr0 > lr0flows >> AT_CAPTURE_FILE([lr0flows]) >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> ]) >> >> AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_dnat;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) >> - table=6 (lr_in_dnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> ]) >> >> AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> - table=1 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.100);) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), > action=(ct_snat(20.0.0.1);) >> - table=1 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), > action=(ct_snat(bef0::1);) >> - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.100);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), > action=(ct_snat(20.0.0.1);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), > action=(ct_snat(bef0::1);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> ]) >> >> check ovn-nbctl --wait=sb lb-add lb2 10.0.0.20:80 10.0.0.40:8080 >> @@ -3280,20 +3335,35 @@ check ovn-nbctl --wait=sb lb-del lb1 >> ovn-sbctl dump-flows lr0 > lr0flows >> >> AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> - table=5 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> - table=5 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.20 && tcp), action=(reg0 = 10.0.0.20; ct_dnat;) >> ]) >> >> AT_CHECK([grep "lr_in_dnat" lr0flows | grep skip_snat_for_lb | sort], > [0], [dnl >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), > action=(flags.skip_snat_for_lb = 1; ct_dnat;) >> - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), > action=(flags.skip_snat_for_lb = 1; ct_lb(backends=10.0.0.40:8080);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.20 && ct_label.natted == 1 && tcp), > action=(flags.skip_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb > = 1; ct_lb(backends=10.0.0.40:8080);) >> ]) >> >> AT_CHECK([grep "lr_out_snat" lr0flows | grep skip_snat_for_lb | sort], > [0], [dnl >> - table=1 (lr_out_snat ), priority=120 , > match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) >> + table=2 (lr_out_snat ), priority=120 , > match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> ]) >> >> AT_CLEANUP >> @@ -3737,3 +3807,451 @@ AT_CHECK([ovn-trace --minimal 'inport == > "sw1-port1" && eth.src == 50:54:00:00:0 >> >> AT_CLEANUP >> ]) >> + >> +OVN_FOR_EACH_NORTHD([ >> +AT_SETUP([ovn -- LR NAT flows]) >> +ovn_start >> + >> +check ovn-nbctl \ >> + -- ls-add sw0 \ >> + -- lb-add lb0 10.0.0.10:80 10.0.0.4:8080 \ >> + -- ls-lb-add sw0 lb0 >> + >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 >> +check ovn-nbctl lsp-add sw0 sw0-lr0 >> +check ovn-nbctl lsp-set-type sw0-lr0 router >> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 >> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 >> + >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> +]) >> + >> +# Create few dnat_and_snat entries >> + >> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.10 10.0.0.0/24 >> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.20 10.0.0.3 >> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.30 10.0.0.10 >> + >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> +]) >> + >> +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 >> + >> +# Create a distributed gw port on lr0 >> +check ovn-nbctl ls-add public >> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 >> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 >> + >> +ovn-nbctl lsp-add public public-lr0 -- set Logical_Switch_Port > public-lr0 \ >> + type=router options:router-port=lr0-public \ >> + -- lsp-set-addresses public-lr0 router >> + >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.10 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.30 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == > 10.0.0.3 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == > 10.0.0.0/24 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == > 10.0.0.10 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == > 10.0.0.3 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +# Associate load balancer to lr0 >> + >> +check ovn-nbctl lb-add lb0 172.168.0.100:8082 "10.0.0.50:82,10.0.0.60:82" >> + >> +# No L4 >> +check ovn-nbctl lb-add lb1 172.168.0.200 "10.0.0.80,10.0.0.81" >> +check ovn-nbctl lb-add lb2 172.168.0.210:60 "10.0.0.50:6062, > 10.0.0.60:6062" udp >> + >> +check ovn-nbctl lr-lb-add lr0 lb0 >> +check ovn-nbctl lr-lb-add lr0 lb1 >> +check ovn-nbctl lr-lb-add lr0 lb2 >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.10 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.30 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20 && inport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && > reg0 == 172.168.0.200 && ct_label.natted == 1 && > is_chassis_resident("cr-lr0-public")), action=(next;) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && > reg0 == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), > action=(ct_lb(backends=10.0.0.80,10.0.0.81);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp && > is_chassis_resident("cr-lr0-public")), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp && > is_chassis_resident("cr-lr0-public")), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.210 && ct_label.natted == 1 && udp && > is_chassis_resident("cr-lr0-public")), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && > is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.4:8080 > );) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.100 && tcp && tcp.dst == 8082 && > is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.210 && udp && udp.dst == 60 && > is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:6062 > ,10.0.0.60:6062);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == > 10.0.0.3 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) >> + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src > == 10.0.0.4 && tcp.src == 8080)) && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) >> + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src > == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) > && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), > action=(ct_dnat;) >> + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src > == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == > 6062)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), > action=(ct_dnat;) >> + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src > == 10.0.0.80) || (ip4.src == 10.0.0.81)) && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == > 10.0.0.0/24 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == > 10.0.0.10 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == > 10.0.0.3 && outport == "lr0-public" && > is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +# Make the logical router as Gateway router >> +check ovn-nbctl clear logical_router_port lr0-public gateway_chassis >> +check ovn-nbctl set logical_router lr0 options:chassis=gw1 >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.20), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.30), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && > reg0 == 172.168.0.200 && ct_label.natted == 1), action=(next;) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && > reg0 == 172.168.0.200), action=(ct_lb(backends=10.0.0.80,10.0.0.81);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), > action=(ct_lb(backends=10.0.0.4:8080);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(ct_lb(backends= > 10.0.0.50:82,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(ct_lb(backends= > 10.0.0.50:6062,10.0.0.60:6062);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == > 10.0.0.0/24), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.10), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.3), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +# Set lb force snat logical router. >> +check ovn-nbctl --wait=sb set logical_router lr0 > options:lb_force_snat_ip="router_ip" >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.20), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.30), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && > reg0 == 172.168.0.200 && ct_label.natted == 1), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && > reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; > ct_lb(backends=10.0.0.80,10.0.0.81);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.210 && udp && udp.dst == 60), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 > ,10.0.0.60:6062);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == > 10.0.0.0/24), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.10), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.3), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +# Add a LB VIP same as router ip. >> +check ovn-nbctl lb-add lb0 172.168.0.10:9082 "10.0.0.50:82,10.0.0.60:82" >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst > == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.20), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.30), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && > reg0 == 172.168.0.200 && ct_label.natted == 1), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && > reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; > ct_lb(backends=10.0.0.80,10.0.0.81);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.210 && udp && udp.dst == 60), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 > ,10.0.0.60:6062);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == > 10.0.0.0/24), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.10), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.3), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +# Add IPv6 router port and LB. >> +check ovn-nbctl lrp-del lr0-sw0 >> +check ovn-nbctl lrp-del lr0-public >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 aef0::1 >> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 > def0::10 >> + >> +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb1) >> +ovn-nbctl set load_balancer $lb1_uuid > vips:'"[[def0::2]]:8000"'='"@<:@aef0::2@:>@:80,@<:@aef0::3@:>@:80"' >> + >> +ovn-nbctl list load_Balancer >> +check ovn-nbctl --wait=sb sync >> + >> +ovn-sbctl dump-flows lr0 > lr0flows >> +AT_CAPTURE_FILE([lr0flows]) >> + >> +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl >> + table=4 (lr_in_unsnat ), priority=0 , match=(1), > action=(next;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-public" && ip6.dst == def0::10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=110 , match=(inport == > "lr0-sw0" && ip6.dst == aef0::1), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst > == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.10), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.20), action=(ct_snat;) >> + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == > 172.168.0.30), action=(ct_snat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl >> + table=5 (lr_in_defrag ), priority=0 , match=(1), > action=(next;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == > 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) >> + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip6.dst == > def0::2 && tcp), action=(xxreg0 = def0::2; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl >> + table=6 (lr_in_dnat ), priority=0 , match=(1), > action=(next;) >> + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == > 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && > reg0 == 172.168.0.200 && ct_label.natted == 1), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && > reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; > ct_lb(backends=10.0.0.80,10.0.0.81);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && > xxreg0 == def0::2 && ct_label.natted == 1 && tcp), > action=(flags.force_snat_for_lb = 1; next;) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb > = 1; ct_lb(backends=10.0.0.4:8080);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82 > ,10.0.0.60:82);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > reg0 == 172.168.0.210 && udp && udp.dst == 60), > action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062 > ,10.0.0.60:6062);) >> + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && > xxreg0 == def0::2 && tcp && tcp.dst == 8000), > action=(flags.force_snat_for_lb = 1; > ct_lb(backends=[[aef0::2]]:80,[[aef0::3]]:80);) >> +]) >> + >> +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl >> + table=0 (lr_out_undnat ), priority=0 , match=(1), > action=(next;) >> + table=0 (lr_out_undnat ), priority=50 , match=(ip), > action=(flags.loopback = 1; ct_dnat;) >> +]) >> + >> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl >> + table=1 (lr_out_post_undnat ), priority=0 , match=(1), > action=(next;) >> + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), > action=(ct_commit { } ; next; ) >> +]) >> + >> +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl >> + table=2 (lr_out_snat ), priority=0 , match=(1), > action=(next;) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), > action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), > action=(ct_snat(10.0.0.1);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-public"), > action=(ct_snat(def0::10);) >> + table=2 (lr_out_snat ), priority=110 , > match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw0"), > action=(ct_snat(aef0::1);) >> + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), > action=(next;) >> + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == > 10.0.0.0/24), action=(ct_snat(172.168.0.10);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.10), action=(ct_snat(172.168.0.30);) >> + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == > 10.0.0.3), action=(ct_snat(172.168.0.20);) >> +]) >> + >> +AT_CLEANUP >> +]) >> diff --git a/tests/ovn.at b/tests/ovn.at >> index bc494fcad9bb..ea1593197f21 100644 >> --- a/tests/ovn.at >> +++ b/tests/ovn.at >> @@ -20571,7 +20571,7 @@ AT_CAPTURE_FILE([sbflows2]) >> OVS_WAIT_FOR_OUTPUT( >> [ovn-sbctl dump-flows > sbflows2 >> ovn-sbctl dump-flows lr0 | grep ct_lb | grep priority=120 | sed > 's/table=..//'], 0, >> - [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && > ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && > is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80, > 20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) >> + [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 > == 10.0.0.10 && tcp && tcp.dst == 80 && > is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80, > 20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) >> ]) >> >> # get the svc monitor mac. >> @@ -20612,8 +20612,8 @@ AT_CHECK( >> AT_CAPTURE_FILE([sbflows4]) >> ovn-sbctl dump-flows lr0 > sbflows4 >> AT_CHECK([grep lr_in_dnat sbflows4 | grep priority=120 | sed > 's/table=..//' | sort], [0], [dnl >> - (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst > == 10.0.0.10 && tcp && tcp.dst == 80 && > is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) >> - (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst > == 10.0.0.10 && tcp && tcp.dst == 80 && > is_chassis_resident("cr-lr0-public")), action=(drop;) >> + (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == > 10.0.0.10 && ct_label.natted == 1 && tcp && > is_chassis_resident("cr-lr0-public")), action=(next;) >> + (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == > 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), > action=(drop;) >> ]) >> >> # Delete sw0-p1 >> diff --git a/tests/system-ovn.at b/tests/system-ovn.at >> index 552fdae52665..4f104171bdba 100644 >> --- a/tests/system-ovn.at >> +++ b/tests/system-ovn.at >> @@ -116,6 +116,7 @@ NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 > 30.0.0.2 | FORMAT_PING], \ >> # Check conntrack entries. >> AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.2) | \ >> sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl >> > +icmp,orig=(src=172.16.1.2,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> >> > icmp,orig=(src=172.16.1.2,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> >> ]) >> >> @@ -298,6 +299,7 @@ NS_CHECK_EXEC([alice1], [ping6 -q -c 3 -i 0.3 -w 2 > fd30::2 | FORMAT_PING], \ >> # Check conntrack entries. >> AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd21::2) | \ >> sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl >> > +icmpv6,orig=(src=fd21::2,dst=fd11::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> >> > icmpv6,orig=(src=fd21::2,dst=fd30::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> >> ]) >> >> @@ -2197,11 +2199,12 @@ > tcp,orig=(src=172.16.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(sr >> ]) >> >> check_est_flows () { >> - n=$(ovs-ofctl dump-flows br-int table=15 | grep \ >> > -"priority=120,ct_state=+est+trk,tcp,metadata=0x2,nw_dst=30.0.0.2,tp_dst=8000" > \ >> -| grep nat | sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') >> + n=$(ovs-ofctl dump-flows br-int table=13 | grep \ >> +"priority=100,tcp,metadata=0x2,nw_dst=30.0.0.2" | grep nat | >> +sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') >> >> echo "n_packets=$n" >> + test ! -z $n >> test "$n" != 0 >> } >> >> @@ -2222,7 +2225,7 @@ ovn-nbctl set load_balancer $uuid vips:'" > 30.0.0.2:8000"'='"192.168.1.2:80,192.16 >> >> ovn-nbctl list load_balancer >> ovn-sbctl dump-flows R2 >> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ >> +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ >> grep 'nat(src=20.0.0.2)']) >> >> dnl Test load-balancing that includes L4 ports in NAT. >> @@ -2260,7 +2263,7 @@ ovn-nbctl set load_balancer $uuid vips:'" > 30.0.0.2:8000"'='"192.168.1.2:80,192.16 >> >> ovn-nbctl list load_balancer >> ovn-sbctl dump-flows R2 >> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ >> +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ >> grep 'nat(src=20.0.0.2)']) >> >> rm -f wget*.log >> -- >> 2.27.0 >> >> >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> >
diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 4074646029b4..d56a121d4d2e 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -2628,39 +2628,9 @@ icmp6 { </li> </ul> - <h3>Ingress Table 4: DEFRAG</h3> - <p> - This is to send packets to connection tracker for tracking and - defragmentation. It contains a priority-0 flow that simply moves traffic - to the next table. - </p> - - <p> - If load balancing rules with virtual IP addresses (and ports) are - configured in <code>OVN_Northbound</code> database for a Gateway router, - a priority-100 flow is added for each configured virtual IP address - <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip - && ip4.dst == <var>VIP</var></code>. For IPv6 <var>VIPs</var>, - the flow matches <code>ip && ip6.dst == <var>VIP</var></code>. - The flow uses the action <code>ct_next;</code> to send IP packets to the - connection tracker for packet de-fragmentation and tracking before - sending it to the next table. - </p> - - <p> - If ECMP routes with symmetric reply are configured in the - <code>OVN_Northbound</code> database for a gateway router, a priority-300 - flow is added for each router port on which symmetric replies are - configured. The matching logic for these ports essentially reverses the - configured logic of the ECMP route. So for instance, a route with a - destination routing policy will instead match if the source IP address - matches the static route's prefix. The flow uses the action - <code>ct_next</code> to send IP packets to the connection tracker for - packet de-fragmentation and tracking before sending it to the next table. - </p> - <h3>Ingress Table 5: UNSNAT</h3> + <h3>Ingress Table 4: UNSNAT</h3> <p> This is for already established connections' reverse traffic. @@ -2669,7 +2639,7 @@ icmp6 { unSNATted here. </p> - <p>Ingress Table 5: UNSNAT on Gateway and Distributed Routers</p> + <p>Ingress Table 4: UNSNAT on Gateway and Distributed Routers</p> <ul> <li> <p> @@ -2696,7 +2666,7 @@ icmp6 { </li> </ul> - <p>Ingress Table 5: UNSNAT on Gateway Routers</p> + <p>Ingress Table 4: UNSNAT on Gateway Routers</p> <ul> <li> @@ -2713,9 +2683,10 @@ icmp6 { <code>lb_force_snat_ip=router_ip</code> then for every logical router port <var>P</var> attached to the Gateway router with the router ip <var>B</var>, a priority-110 flow is added with the match - <code>inport == <var>P</var> && ip4.dst == <var>B</var></code> or - <code>inport == <var>P</var> && ip6.dst == <var>B</var></code> - with an action <code>ct_snat; </code>. + <code>inport == <var>P</var> && + ip4.dst == <var>B</var></code> or <code>inport == <var>P</var> + && ip6.dst == <var>B</var></code> with an action + <code>ct_snat; </code>. </p> <p> @@ -2745,7 +2716,7 @@ icmp6 { </li> </ul> - <p>Ingress Table 5: UNSNAT on Distributed Routers</p> + <p>Ingress Table 4: UNSNAT on Distributed Routers</p> <ul> <li> @@ -2776,6 +2747,40 @@ icmp6 { </li> </ul> + <h3>Ingress Table 5: DEFRAG</h3> + + <p> + This is to send packets to connection tracker for tracking and + defragmentation. It contains a priority-0 flow that simply moves traffic + to the next table. + </p> + + <p> + If load balancing rules with virtual IP addresses (and ports) are + configured in <code>OVN_Northbound</code> database for a Gateway router, + a priority-100 flow is added for each configured virtual IP address + <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches <code>ip + && ip4.dst == <var>VIP</var></code>. For IPv6 <var>VIPs</var>, + the flow matches <code>ip && ip6.dst == <var>VIP</var></code>. + The flow applies the action <code>reg0 = <var>VIP</var> + && ct_dnat;</code> to send IP packets to the + connection tracker for packet de-fragmentation and to dnat the + destination IP for the committed connection before sending it to the + next table. + </p> + + <p> + If ECMP routes with symmetric reply are configured in the + <code>OVN_Northbound</code> database for a gateway router, a priority-300 + flow is added for each router port on which symmetric replies are + configured. The matching logic for these ports essentially reverses the + configured logic of the ECMP route. So for instance, a route with a + destination routing policy will instead match if the source IP address + matches the static route's prefix. The flow uses the action + <code>ct_next</code> to send IP packets to the connection tracker for + packet de-fragmentation and tracking before sending it to the next table. + </p> + <h3>Ingress Table 6: DNAT</h3> <p> @@ -2828,19 +2833,28 @@ icmp6 { </li> <li> - For all the configured load balancing rules for a router in - <code>OVN_Northbound</code> database that includes a L4 port - <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 address - <var>VIP</var>, a priority-120 flow that matches on - <code>ct.est && ip && ip4.dst == <var>VIP</var> - && <var>P</var> && <var>P</var>.dst == <var>PORT - </var></code> (<code>ip6.dst == <var>VIP</var></code> in the IPv6 case) - with an action of <code>ct_dnat;</code>. If the router is - configured to force SNAT any load-balanced packets, the above action - will be replaced by <code>flags.force_snat_for_lb = 1; ct_dnat;</code>. - If the load balancing rule is configured with <code>skip_snat</code> - set to true, the above action will be replaced by - <code>flags.skip_snat_for_lb = 1; ct_dnat;</code>. + <p> + For all the configured load balancing rules for a router in + <code>OVN_Northbound</code> database that includes a L4 port + <var>PORT</var> of protocol <var>P</var> and IPv4 or IPv6 address + <var>VIP</var>, a priority-120 flow that matches on + <code>ct.est && ip && reg0 == <var>VIP</var> + && <var>P</var> && <var>P</var>.dst == <var>PORT + </var></code> (<code>xxreg0 == <var>VIP</var></code> in the + IPv6 case) with an action of <code>next;</code>. If the router is + configured to force SNAT any load-balanced packets, the above action + will be replaced by <code>flags.force_snat_for_lb = 1; next;</code>. + If the load balancing rule is configured with <code>skip_snat</code> + set to true, the above action will be replaced by + <code>flags.skip_snat_for_lb = 1; next;</code>. + </p> + + <p> + Previous table <code>lr_in_defrag</code> sets the register + <code>reg0</code> (or <code>xxreg0</code> for IPv6) and does + <code>ct_dnat</code>. Hence for established traffic, this + table just advances the packet to the next stage. + </p> </li> <li> @@ -3876,7 +3890,26 @@ nd_ns { </li> </ul> - <h3>Egress Table 1: SNAT</h3> + <h3>Egress Table 1: Post UNDNAT on Gateway Routers</h3> + + <p> + <ul> + <li> + A priority-50 logical flow is added that commits any untracked flows + from the previous table <code>lr_out_undnat</code>. This flow + matches on <code>ct.new && ip</code> with action + <code>ct_commit { } ; next; </code>. + </li> + + <li> + A priority-0 logical flow with match <code>1</code> has actions + <code>next;</code>. + </li> + + </ul> + </p> + + <h3>Egress Table 2: SNAT</h3> <p> Packets that are configured to be SNATed get their source IP address @@ -3892,7 +3925,7 @@ nd_ns { </li> </ul> - <p>Egress Table 1: SNAT on Gateway Routers</p> + <p>Egress Table 2: SNAT on Gateway Routers</p> <ul> <li> @@ -3991,7 +4024,7 @@ nd_ns { </li> </ul> - <p>Egress Table 1: SNAT on Distributed Routers</p> + <p>Egress Table 2: SNAT on Distributed Routers</p> <ul> <li> @@ -4051,7 +4084,7 @@ nd_ns { </li> </ul> - <h3>Egress Table 2: Egress Loopback</h3> + <h3>Egress Table 3: Egress Loopback</h3> <p> For distributed logical routers where one of the logical router @@ -4120,7 +4153,7 @@ clone { </li> </ul> - <h3>Egress Table 3: Delivery</h3> + <h3>Egress Table 4: Delivery</h3> <p> Packets that reach this table are ready for delivery. It contains: diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index d97ab4a5b39c..27e5fbea9f4f 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -187,8 +187,8 @@ enum ovn_stage { PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 4, "lr_in_defrag") \ - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 5, "lr_in_unsnat") \ + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ PIPELINE_STAGE(ROUTER, IN, DNAT, 6, "lr_in_dnat") \ PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 7, "lr_in_ecmp_stateful") \ PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 8, "lr_in_nd_ra_options") \ @@ -204,10 +204,11 @@ enum ovn_stage { PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 18, "lr_in_arp_request") \ \ /* Logical router egress stages. */ \ - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, SNAT, 1, "lr_out_snat") \ - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 2, "lr_out_egr_loop") \ - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 3, "lr_out_delivery") + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 1, "lr_out_post_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, SNAT, 2, "lr_out_snat") \ + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 3, "lr_out_egr_loop") \ + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 4, "lr_out_delivery") #define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ S_##DP_TYPE##_##PIPELINE##_##STAGE \ @@ -643,6 +644,12 @@ struct ovn_datapath { /* Multicast data. */ struct mcast_info mcast_info; + /* Applies to only logical router datapath. + * True if logical router is a gateway router. i.e options:chassis is set. + * If this is true, then 'l3dgw_port' and 'l3redirect_port' will be + * ignored. */ + bool is_gw_router; + /* OVN northd only needs to know about the logical router gateway port for * NAT on a distributed router. This "distributed gateway port" is * populated only when there is a gateway chassis specified for one of @@ -1247,6 +1254,9 @@ join_datapaths(struct northd_context *ctx, struct hmap *datapaths, init_mcast_info_for_datapath(od); init_nat_entries(od); init_lb_ips(od); + if (smap_get(&od->nbr->options, "chassis")) { + od->is_gw_router = true; + } ovs_list_push_back(lr_list, &od->lr_list); } } @@ -8731,20 +8741,33 @@ add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od, } /* A match and actions for established connections. */ - char *est_match = xasprintf("ct.est && %s", ds_cstr(match)); + struct ds est_match = DS_EMPTY_INITIALIZER; + ds_put_format(&est_match, + "ct.est && ip && %sreg0 == %s && ct_label.natted == 1", + IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) ? "" : "xx", + lb_vip->vip_str); + if (lb_vip->vip_port) { + ds_put_format(&est_match, " && %s", proto); + } + if (od->l3redirect_port && + (lb_vip->n_backends || !lb_vip->empty_backend_rej)) { + ds_put_format(&est_match, " && is_chassis_resident(%s)", + od->l3redirect_port->json_key); + } if (snat_type == FORCE_SNAT || snat_type == SKIP_SNAT) { - char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; ct_dnat;", + char *est_actions = xasprintf("flags.%s_snat_for_lb = 1; next;", snat_type == SKIP_SNAT ? "skip" : "force"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, - est_match, est_actions, &lb->header_); + ds_cstr(&est_match), est_actions, + &lb->header_); free(est_actions); } else { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, priority, - est_match, "ct_dnat;", &lb->header_); + ds_cstr(&est_match), "next;", &lb->header_); } free(new_match); - free(est_match); + ds_destroy(&est_match); const char *ip_match = NULL; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { @@ -8829,8 +8852,8 @@ add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, struct hmap *lbs, struct shash *meter_groups, - struct sset *nat_entries, struct ds *match, - struct ds *actions) + struct sset *nat_entries, + struct ds *match, struct ds *actions) { /* A set to hold all ips that need defragmentation and tracking. */ struct sset all_ips = SSET_INITIALIZER(&all_ips); @@ -8852,10 +8875,17 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, for (size_t j = 0; j < lb->n_vips; j++) { struct ovn_lb_vip *lb_vip = &lb->vips[j]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[j]; + + bool is_udp = nullable_string_is_equal(nb_lb->protocol, "udp"); + bool is_sctp = nullable_string_is_equal(nb_lb->protocol, + "sctp"); + const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : "tcp"; + ds_clear(actions); build_lb_vip_actions(lb_vip, lb_vip_nb, actions, lb->selection_fields, false); + struct ds defrag_actions = DS_EMPTY_INITIALIZER; if (!sset_contains(&all_ips, lb_vip->vip_str)) { sset_add(&all_ips, lb_vip->vip_str); /* If there are any load balancing rules, we should send @@ -8867,17 +8897,28 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, * 2. If there are L4 ports in load balancing rules, we * need the defragmentation to match on L4 ports. */ ds_clear(match); + ds_clear(&defrag_actions); if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(match, "ip && ip4.dst == %s", lb_vip->vip_str); + ds_put_format(&defrag_actions, "reg0 = %s; ct_dnat;", + lb_vip->vip_str); } else { ds_put_format(match, "ip && ip6.dst == %s", lb_vip->vip_str); + ds_put_format(&defrag_actions, "xxreg0 = %s; ct_dnat;", + lb_vip->vip_str); + } + + if (lb_vip->vip_port) { + ds_put_format(match, " && %s", proto); } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, - 100, ds_cstr(match), "ct_next;", + 100, ds_cstr(match), + ds_cstr(&defrag_actions), &nb_lb->header_); } + ds_destroy(&defrag_actions); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows @@ -8886,18 +8927,14 @@ build_lrouter_lb_flows(struct hmap *lflows, struct ovn_datapath *od, * flow is for ct.est with an action of "ct_dnat;". */ ds_clear(match); if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { - ds_put_format(match, "ip && ip4.dst == %s", + ds_put_format(match, "ip && reg0 == %s", lb_vip->vip_str); } else { - ds_put_format(match, "ip && ip6.dst == %s", + ds_put_format(match, "ip && xxreg0 == %s", lb_vip->vip_str); } int prio = 110; - bool is_udp = nullable_string_is_equal(nb_lb->protocol, "udp"); - bool is_sctp = nullable_string_is_equal(nb_lb->protocol, - "sctp"); - const char *proto = is_udp ? "udp" : is_sctp ? "sctp" : "tcp"; if (lb_vip->vip_port) { ds_put_format(match, " && %s && %s.dst == %d", proto, @@ -11400,8 +11437,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, * part of a reply. We undo the DNAT here. * * Note that this only applies for NAT on a distributed router. - * Undo DNAT on a gateway router is done in the ingress DNAT - * pipeline stage. */ + */ if (!od->l3dgw_port || (strcmp(nat->type, "dnat") && strcmp(nat->type, "dnat_and_snat"))) { return; @@ -11681,9 +11717,22 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + /* For Gateway routers, if the gateway router has load balancer or DNAT + * rules, we commit newly initiated connections in the reply direction + * to the DNAT zone. This ensures that these flows are tracked. If the flow + * was not committed, it would produce ongoing datapath flows with the + * ct.new flag set. Some NICs are unable to offload these flows. + */ + if (od->is_gw_router && + (od->nbr->n_nat || od->nbr->n_load_balancer)) { + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, + "ip && ct.new", "ct_commit { } ; next; "); + } + /* Send the IPv6 NS packets to next table. When ovn-controller * generates IPv6 NS (for the action - nd_ns{}), the injected * packet would go through conntrack - which is not required. */ @@ -11848,18 +11897,12 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); } } - - /* For gateway router, re-circulate every packet through - * the DNAT zone. This helps with the following. - * - * Any packet that needs to be unDNATed in the reverse - * direction gets unDNATed. Ideally this could be done in - * the egress pipeline. But since the gateway router - * does not have any feature that depends on the source - * ip address being external IP address for IP routing, - * we can do it here, saving a future re-circulation. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, - "ip", "flags.loopback = 1; ct_dnat;"); + /* For gateway router, re-circulate every packet through the DNAT zone + * so that packets that need to be unDNATed in the reverse direction + * get unDNATed. + */ + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, + "ip", "flags.loopback = 1; ct_dnat;"); } /* Load balancing and packet defrag are only valid on diff --git a/northd/ovn_northd.dl b/northd/ovn_northd.dl index 3afa80a3b549..727641ac6ae1 100644 --- a/northd/ovn_northd.dl +++ b/northd/ovn_northd.dl @@ -1454,8 +1454,8 @@ function s_ROUTER_IN_ADMISSION(): Stage { Stage{Ingress, 0, "lr_in_admiss function s_ROUTER_IN_LOOKUP_NEIGHBOR(): Stage { Stage{Ingress, 1, "lr_in_lookup_neighbor"} } function s_ROUTER_IN_LEARN_NEIGHBOR(): Stage { Stage{Ingress, 2, "lr_in_learn_neighbor"} } function s_ROUTER_IN_IP_INPUT(): Stage { Stage{Ingress, 3, "lr_in_ip_input"} } -function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 4, "lr_in_defrag"} } -function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 5, "lr_in_unsnat"} } +function s_ROUTER_IN_UNSNAT(): Stage { Stage{Ingress, 4, "lr_in_unsnat"} } +function s_ROUTER_IN_DEFRAG(): Stage { Stage{Ingress, 5, "lr_in_defrag"} } function s_ROUTER_IN_DNAT(): Stage { Stage{Ingress, 6, "lr_in_dnat"} } function s_ROUTER_IN_ECMP_STATEFUL(): Stage { Stage{Ingress, 7, "lr_in_ecmp_stateful"} } function s_ROUTER_IN_ND_RA_OPTIONS(): Stage { Stage{Ingress, 8, "lr_in_nd_ra_options"} } @@ -1472,9 +1472,10 @@ function s_ROUTER_IN_ARP_REQUEST(): Stage { Stage{Ingress, 18, "lr_in_arp_re /* Logical router egress stages. */ function s_ROUTER_OUT_UNDNAT(): Stage { Stage{ Egress, 0, "lr_out_undnat"} } -function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 1, "lr_out_snat"} } -function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 2, "lr_out_egr_loop"} } -function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 3, "lr_out_delivery"} } +function s_ROUTER_OUT_POST_UNDNAT(): Stage { Stage{ Egress, 1, "lr_out_post_undnat"} } +function s_ROUTER_OUT_SNAT(): Stage { Stage{ Egress, 2, "lr_out_snat"} } +function s_ROUTER_OUT_EGR_LOOP(): Stage { Stage{ Egress, 3, "lr_out_egr_loop"} } +function s_ROUTER_OUT_DELIVERY(): Stage { Stage{ Egress, 4, "lr_out_delivery"} } /* * OVS register usage: @@ -2886,7 +2887,8 @@ for (&Switch(._uuid = ls_uuid)) { function get_match_for_lb_key(ip_address: v46_ip, port: bit<16>, protocol: Option<string>, - redundancy: bool): string = { + redundancy: bool, + use_nexthop_reg: bool): string = { var port_match = if (port != 0) { var proto = if (protocol == Some{"udp"}) { "udp" @@ -2900,8 +2902,18 @@ function get_match_for_lb_key(ip_address: v46_ip, }; var ip_match = match (ip_address) { - IPv4{ipv4} -> "ip4.dst == ${ipv4}", - IPv6{ipv6} -> "ip6.dst == ${ipv6}" + IPv4{ipv4} -> + if (use_nexthop_reg) { + "${rEG_NEXT_HOP()} == ${ipv4}" + } else { + "ip4.dst == ${ipv4}" + }, + IPv6{ipv6} -> + if (use_nexthop_reg) { + "xx${rEG_NEXT_HOP()} == ${ipv6}" + } else { + "ip6.dst == ${ipv6}" + } }; if (redundancy) { "ip && " } else { "" } ++ ip_match ++ port_match @@ -2935,7 +2947,11 @@ function build_lb_vip_actions(lbvip: Intern<LBVIPWithStatus>, for (pair in lbvip.backends) { (var backend, var up) = pair; if (up) { - up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") + if (backend.port != 0) { + up_backends.insert("${backend.ip.to_bracketed_string()}:${backend.port}") + } else { + up_backends.insert("${backend.ip.to_bracketed_string()}") + } } }; @@ -2981,7 +2997,7 @@ Flow(.logical_datapath = sw._uuid, build_lb_vip_actions(lbvip, s_SWITCH_OUT_QOS_MARK(), actions0 ++ actions1) }, - var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, false). + var __match = "ct.new && " ++ get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, false, false). /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. @@ -3019,7 +3035,7 @@ for (&Switch(._uuid = ls_uuid, .has_lb_vip = true)) { .__match = "ip && ct.new && ct.trk && ${rEGBIT_HAIRPIN()} == 1", .actions = "ct_snat_to_vip; next;", .external_ids = stage_hint(ls_uuid)); - + /* If packet needs to be hairpinned, for established sessions there * should already be an SNAT conntrack entry. */ @@ -5379,13 +5395,14 @@ function default_allow_flow(datapath: uuid, stage: Stage): Flow { .actions = "next;", .external_ids = map_empty()} } -for (&Router(._uuid = lr_uuid)) { +for (r in &Router(._uuid = lr_uuid)) { /* Packets are allowed by default. */ Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DEFRAG())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_UNSNAT())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_SNAT())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_DNAT())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_UNDNAT())]; + Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_POST_UNDNAT())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_OUT_EGR_LOOP())]; Flow[default_allow_flow(lr_uuid, s_ROUTER_IN_ECMP_STATEFUL())]; @@ -5400,6 +5417,25 @@ for (&Router(._uuid = lr_uuid)) { .external_ids = map_empty()) } +for (r in &Router(._uuid = lr_uuid, + .is_gateway = is_gateway, + .nat = nat, + .load_balancer = load_balancer) + if is_gateway and (not is_empty(nat) or not is_empty(load_balancer))) { + /* For Gateway routers, if the gateway router has load balancer or DNAT + * rules, we commit newly initiated connections in the reply direction + * to the DNAT zone. This ensures that these flows are tracked. If the flow + * was not committed, it would produce ongoing datapath flows with the + * ct.new flag set. Some NICs are unable to offload these flows. + */ + Flow(.logical_datapath = lr_uuid, + .stage = s_ROUTER_OUT_POST_UNDNAT(), + .priority = 50, + .__match = "ip && ct.new", + .actions = "ct_commit { } ; next; ", + .external_ids = map_empty()) +} + Flow(.logical_datapath = lr, .stage = s_ROUTER_OUT_SNAT(), .priority = 120, @@ -5438,7 +5474,7 @@ function lrouter_nat_add_ext_ip_match( Some{AllowedExtIps{__as}} -> (" && ${ipX}.${dir} == $${__as.name}", None), Some{ExemptedExtIps{__as}} -> { /* Priority of logical flows corresponding to exempted_ext_ips is - * +1 of the corresponding regulr NAT rule. + * +1 of the corresponding regular NAT rule. * For example, if we have following NAT rule and we associate * exempted external ips to it: * "ovn-nbctl lr-nat-add router dnat_and_snat 10.15.24.139 50.0.0.11" @@ -5746,8 +5782,7 @@ for (r in &Router(._uuid = lr_uuid, * part of a reply. We undo the DNAT here. * * Note that this only applies for NAT on a distributed router. - * Undo DNAT on a gateway router is done in the ingress DNAT - * pipeline stage. */ + */ if ((nat.nat.__type == "dnat" or nat.nat.__type == "dnat_and_snat")) { Some{var gwport} = l3dgw_port in var __match = @@ -5953,16 +5988,11 @@ for (r in &Router(._uuid = lr_uuid, .context = "lb"); /* For gateway router, re-circulate every packet through - * the DNAT zone. This helps with the following. - * - * Any packet that needs to be unDNATed in the reverse - * direction gets unDNATed. Ideally this could be done in - * the egress pipeline. But since the gateway router - * does not have any feature that depends on the source - * ip address being external IP address for IP routing, - * we can do it here, saving a future re-circulation. */ + * the DNAT zone so that packets that need to be unDNATed in the reverse + * direction get unDNATed. + */ Flow(.logical_datapath = lr_uuid, - .stage = s_ROUTER_IN_DNAT(), + .stage = s_ROUTER_OUT_UNDNAT(), .priority = 50, .__match = "ip", .actions = "flags.loopback = 1; ct_dnat;", @@ -6024,7 +6054,16 @@ for (RouterLBVIP( * pick a DNAT ip address from a group. * 2. If there are L4 ports in load balancing rules, we * need the defragmentation to match on L4 ports. */ - var __match = "ip && ${ipX}.dst == ${ip_address}" in + var match1 = "ip && ${ipX}.dst == ${ip_address}" in + var match2 = + if (port != 0) { + " && ${proto}" + } else { + "" + } in + var __match = match1 ++ match2 in + var xx = ip_address.xxreg() in + var __actions = "${xx}${rEG_NEXT_HOP()} = ${ip_address}; ct_dnat;" in /* One of these flows must be created for each unique LB VIP address. * We create one for each VIP:port pair; flows with the same IP and * different port numbers will produce identical flows that will @@ -6033,7 +6072,7 @@ for (RouterLBVIP( .stage = s_ROUTER_IN_DEFRAG(), .priority = 100, .__match = __match, - .actions = "ct_next;", + .actions = __actions, .external_ids = stage_hint(lb._uuid)); /* Higher priority rules are added for load-balancing in DNAT @@ -6041,7 +6080,8 @@ for (RouterLBVIP( * via add_router_lb_flow(). One flow is for specific matching * on ct.new with an action of "ct_lb($targets);". The other * flow is for ct.est with an action of "ct_dnat;". */ - var match1 = "ip && ${ipX}.dst == ${ip_address}" in + var xx = ip_address.xxreg() in + var match1 = "ip && ${xx}${rEG_NEXT_HOP()} == ${ip_address}" in (var prio, var match2) = if (port != 0) { (120, " && ${proto} && ${proto}.dst == ${port}") @@ -6056,12 +6096,21 @@ for (RouterLBVIP( var snat_for_lb = snat_for_lb(r.options, lb) in { /* A match and actions for established connections. */ - var est_match = "ct.est && " ++ __match in + var est_match = "ct.est && " ++ match1 ++ " && ct_label.natted == 1" ++ + if (port != 0) { + " && ${proto}" + } else { + "" + } ++ + match ((l3dgw_port, backends != "" or lb.options.get_bool_def("reject", false))) { + (Some{gwport}, true) -> " && is_chassis_resident(${redirect_port_name})", + _ -> "" + } in var actions = match (snat_for_lb) { - SkipSNAT -> "flags.skip_snat_for_lb = 1; ct_dnat;", - ForceSNAT -> "flags.force_snat_for_lb = 1; ct_dnat;", - _ -> "ct_dnat;" + SkipSNAT -> "flags.skip_snat_for_lb = 1; next;", + ForceSNAT -> "flags.force_snat_for_lb = 1; next;", + _ -> "next;" } in Flow(.logical_datapath = lr_uuid, .stage = s_ROUTER_IN_DNAT(), @@ -6152,7 +6201,7 @@ Flow(.logical_datapath = r._uuid, r.load_balancer.contains(lb._uuid), var __match = "ct.new && " ++ - get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, true) ++ + get_match_for_lb_key(lbvip.vip_addr, lbvip.vip_port, lb.protocol, true, true) ++ match (r.l3dgw_port) { Some{gwport} -> " && is_chassis_resident(${r.redirect_port_name})", _ -> "" diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index d81975cb18a6..accaa87033f4 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -1406,40 +1406,39 @@ AT_SETUP([ovn -- Load balancer VIP in NAT entries]) AT_SKIP_IF([test $HAVE_PYTHON = no]) ovn_start -ovn-nbctl lr-add lr0 -ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 -ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 +check ovn-nbctl lr-add lr0 +check ovn-nbctl lrp-add lr0 lr0-public 00:00:01:01:02:04 192.168.2.1/24 +check ovn-nbctl lrp-add lr0 lr0-join 00:00:01:01:02:04 10.10.0.1/24 -ovn-nbctl set logical_router lr0 options:chassis=ch1 +check ovn-nbctl set logical_router lr0 options:chassis=ch1 -ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" -ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp -ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" -ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" +check ovn-nbctl lb-add lb1 "192.168.2.1:8080" "10.0.0.4:8080" +check ovn-nbctl lb-add lb2 "192.168.2.4:8080" "10.0.0.5:8080" udp +check ovn-nbctl lb-add lb3 "192.168.2.5:8080" "10.0.0.6:8080" +check ovn-nbctl lb-add lb4 "192.168.2.6:8080" "10.0.0.7:8080" -ovn-nbctl lr-lb-add lr0 lb1 -ovn-nbctl lr-lb-add lr0 lb2 -ovn-nbctl lr-lb-add lr0 lb3 -ovn-nbctl lr-lb-add lr0 lb4 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 +check ovn-nbctl lr-lb-add lr0 lb3 +check ovn-nbctl lr-lb-add lr0 lb4 -ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 -ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 +check ovn-nbctl lr-nat-add lr0 snat 192.168.2.1 10.0.0.0/24 +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.2.4 10.0.0.4 check ovn-nbctl --wait=sb lr-nat-add lr0 dnat 192.168.2.5 10.0.0.5 ovn-sbctl dump-flows lr0 > sbflows AT_CAPTURE_FILE([sbflows]) -OVS_WAIT_UNTIL([test 1 = $(grep lr_in_unsnat sbflows | \ -grep "ip4 && ip4.dst == 192.168.2.1 && tcp && tcp.dst == 8080" -c) ]) - -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ -grep "ip4 && ip4.dst == 192.168.2.4 && udp && udp.dst == 8080" -c) ]) - -AT_CHECK([test 1 = $(grep lr_in_unsnat sbflows | \ -grep "ip4 && ip4.dst == 192.168.2.5 && tcp && tcp.dst == 8080" -c) ]) - -AT_CHECK([test 0 = $(grep lr_in_unsnat sbflows | \ -grep "ip4 && ip4.dst == 192.168.2.6 && tcp && tcp.dst == 8080" -c) ]) +# There shoule be no flows for LB VIPs in lr_in_unsnat if the VIP is not a +# dnat_and_snat or snat entry. +AT_CHECK([grep "lr_in_unsnat" sbflows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.1 && tcp && tcp.dst == 8080), action=(next;) + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.4 && udp && udp.dst == 8080), action=(next;) + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 192.168.2.5 && tcp && tcp.dst == 8080), action=(next;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 192.168.2.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 192.168.2.4), action=(ct_snat;) +]) AT_CLEANUP ]) @@ -1458,8 +1457,8 @@ ovn-nbctl set logical_router lr0 options:dnat_force_snat_ip=192.168.2.3 ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list lr0 | grep lr_in_unsnat | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 192.168.2.3), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 192.168.2.3), action=(ct_snat;) ]) AT_CLEANUP @@ -3163,14 +3162,28 @@ ovn-sbctl dump-flows lr0 > lr0flows AT_CAPTURE_FILE([lr0flows]) AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_dnat;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) ]) check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="20.0.0.4 aef0::4" @@ -3180,23 +3193,37 @@ AT_CAPTURE_FILE([lr0flows]) AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) - table=5 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 20.0.0.4), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst == aef0::4), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(ip4 && ip4.dst == 20.0.0.4), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(ip6 && ip6.dst == aef0::4), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) - table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) - table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) + table=2 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) ]) check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="router_ip" @@ -3208,25 +3235,39 @@ AT_CHECK([grep "lr_in_ip_input" lr0flows | grep "priority=60" | sort], [0], [dnl ]) AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) ]) check ovn-nbctl --wait=sb remove logical_router lr0 options chassis @@ -3235,12 +3276,12 @@ ovn-sbctl dump-flows lr0 > lr0flows AT_CAPTURE_FILE([lr0flows]) AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) ]) check ovn-nbctl set logical_router lr0 options:chassis=ch1 @@ -3250,27 +3291,41 @@ ovn-sbctl dump-flows lr0 > lr0flows AT_CAPTURE_FILE([lr0flows]) AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) ]) AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_dnat;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) - table=6 (lr_in_dnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl - table=1 (lr_out_snat ), priority=0 , match=(1), action=(next;) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) - table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), action=(ct_snat(bef0::1);) - table=1 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), action=(ct_snat(bef0::1);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) ]) check ovn-nbctl --wait=sb lb-add lb2 10.0.0.20:80 10.0.0.40:8080 @@ -3280,20 +3335,35 @@ check ovn-nbctl --wait=sb lb-del lb1 ovn-sbctl dump-flows lr0 > lr0flows AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl - table=5 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) - table=5 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.100), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip4.dst == 20.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw1" && ip6.dst == bef0::1), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.20 && tcp), action=(reg0 = 10.0.0.20; ct_dnat;) ]) AT_CHECK([grep "lr_in_dnat" lr0flows | grep skip_snat_for_lb | sort], [0], [dnl - table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_dnat;) - table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_lb(backends=10.0.0.40:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.20 && ct_label.natted == 1 && tcp), action=(flags.skip_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.20 && tcp && tcp.dst == 80), action=(flags.skip_snat_for_lb = 1; ct_lb(backends=10.0.0.40:8080);) ]) AT_CHECK([grep "lr_out_snat" lr0flows | grep skip_snat_for_lb | sort], [0], [dnl - table=1 (lr_out_snat ), priority=120 , match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(flags.skip_snat_for_lb == 1 && ip), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) ]) AT_CLEANUP @@ -3737,3 +3807,451 @@ AT_CHECK([ovn-trace --minimal 'inport == "sw1-port1" && eth.src == 50:54:00:00:0 AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn -- LR NAT flows]) +ovn_start + +check ovn-nbctl \ + -- ls-add sw0 \ + -- lb-add lb0 10.0.0.10:80 10.0.0.4:8080 \ + -- ls-lb-add sw0 lb0 + +check ovn-nbctl lr-add lr0 +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +check ovn-nbctl lsp-add sw0 sw0-lr0 +check ovn-nbctl lsp-set-type sw0-lr0 router +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 + +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) +]) + +# Create few dnat_and_snat entries + +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.10 10.0.0.0/24 +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.20 10.0.0.3 +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.30 10.0.0.10 + +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) +]) + +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 + +# Create a distributed gw port on lr0 +check ovn-nbctl ls-add public +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 + +ovn-nbctl lsp-add public public-lr0 -- set Logical_Switch_Port public-lr0 \ + type=router options:router-port=lr0-public \ + -- lsp-set-addresses public-lr0 router + +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) +]) + +# Associate load balancer to lr0 + +check ovn-nbctl lb-add lb0 172.168.0.100:8082 "10.0.0.50:82,10.0.0.60:82" + +# No L4 +check ovn-nbctl lb-add lb1 172.168.0.200 "10.0.0.80,10.0.0.81" +check ovn-nbctl lb-add lb2 172.168.0.210:60 "10.0.0.50:6062,10.0.0.60:6062" udp + +check ovn-nbctl lr-lb-add lr0 lb0 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.30 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);) + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1 && is_chassis_resident("cr-lr0-public")), action=(next;) + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.80,10.0.0.81);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp && is_chassis_resident("cr-lr0-public")), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.4:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.50:6062,10.0.0.60:6062);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=100 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.4 && tcp.src == 8080)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && tcp.src == 82) || (ip4.src == 10.0.0.60 && tcp.src == 82)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.50 && udp.src == 6062) || (ip4.src == 10.0.0.60 && udp.src == 6062)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) + table=0 (lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src == 10.0.0.80) || (ip4.src == 10.0.0.81)) && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=153 , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.10 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=161 , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat(172.168.0.20);) +]) + +# Make the logical router as Gateway router +check ovn-nbctl clear logical_router_port lr0-public gateway_chassis +check ovn-nbctl set logical_router lr0 options:chassis=gw1 +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(next;) + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(ct_lb(backends=10.0.0.80,10.0.0.81);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(ct_lb(backends=10.0.0.4:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(ct_lb(backends=10.0.0.50:6062,10.0.0.60:6062);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) +]) + +# Set lb force snat logical router. +check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="router_ip" +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062,10.0.0.60:6062);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) +]) + +# Add a LB VIP same as router ip. +check ovn-nbctl lb-add lb0 172.168.0.10:9082 "10.0.0.50:82,10.0.0.60:82" +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062,10.0.0.60:6062);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) +]) + +# Add IPv6 router port and LB. +check ovn-nbctl lrp-del lr0-sw0 +check ovn-nbctl lrp-del lr0-public +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 aef0::1 +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24 def0::10 + +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb1) +ovn-nbctl set load_balancer $lb1_uuid vips:'"[[def0::2]]:8000"'='"@<:@aef0::2@:>@:80,@<:@aef0::3@:>@:80"' + +ovn-nbctl list load_Balancer +check ovn-nbctl --wait=sb sync + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_in_unsnat" lr0flows | sort], [0], [dnl + table=4 (lr_in_unsnat ), priority=0 , match=(1), action=(next;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-public" && ip6.dst == def0::10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip4.dst == 10.0.0.1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=110 , match=(inport == "lr0-sw0" && ip6.dst == aef0::1), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=120 , match=(ip4 && ip4.dst == 172.168.0.10 && tcp && tcp.dst == 9082), action=(next;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.10), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.20), action=(ct_snat;) + table=4 (lr_in_unsnat ), priority=90 , match=(ip && ip4.dst == 172.168.0.30), action=(ct_snat;) +]) + +AT_CHECK([grep "lr_in_defrag" lr0flows | sort], [0], [dnl + table=5 (lr_in_defrag ), priority=0 , match=(1), action=(next;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 10.0.0.10 && tcp), action=(reg0 = 10.0.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.10 && tcp), action=(reg0 = 172.168.0.10; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.100 && tcp), action=(reg0 = 172.168.0.100; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.200), action=(reg0 = 172.168.0.200; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip4.dst == 172.168.0.210 && udp), action=(reg0 = 172.168.0.210; ct_dnat;) + table=5 (lr_in_defrag ), priority=100 , match=(ip && ip6.dst == def0::2 && tcp), action=(xxreg0 = def0::2; ct_dnat;) +]) + +AT_CHECK([grep "lr_in_dnat" lr0flows | sort], [0], [dnl + table=6 (lr_in_dnat ), priority=0 , match=(1), action=(next;) + table=6 (lr_in_dnat ), priority=100 , match=(ip && ip4.dst == 172.168.0.20), action=(flags.loopback = 1; ct_dnat(10.0.0.3);) + table=6 (lr_in_dnat ), priority=110 , match=(ct.est && ip && reg0 == 172.168.0.200 && ct_label.natted == 1), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=110 , match=(ct.new && ip && reg0 == 172.168.0.200), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.80,10.0.0.81);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.10 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.100 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 172.168.0.210 && ct_label.natted == 1 && udp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.est && ip && xxreg0 == def0::2 && ct_label.natted == 1 && tcp), action=(flags.force_snat_for_lb = 1; next;) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.4:8080);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.10 && tcp && tcp.dst == 9082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.100 && tcp && tcp.dst == 8082), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:82,10.0.0.60:82);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 172.168.0.210 && udp && udp.dst == 60), action=(flags.force_snat_for_lb = 1; ct_lb(backends=10.0.0.50:6062,10.0.0.60:6062);) + table=6 (lr_in_dnat ), priority=120 , match=(ct.new && ip && xxreg0 == def0::2 && tcp && tcp.dst == 8000), action=(flags.force_snat_for_lb = 1; ct_lb(backends=[[aef0::2]]:80,[[aef0::3]]:80);) +]) + +AT_CHECK([grep "lr_out_undnat" lr0flows | sort], [0], [dnl + table=0 (lr_out_undnat ), priority=0 , match=(1), action=(next;) + table=0 (lr_out_undnat ), priority=50 , match=(ip), action=(flags.loopback = 1; ct_dnat;) +]) + +AT_CHECK([grep "lr_out_post_undnat" lr0flows | sort], [0], [dnl + table=1 (lr_out_post_undnat ), priority=0 , match=(1), action=(next;) + table=1 (lr_out_post_undnat ), priority=50 , match=(ip && ct.new), action=(ct_commit { } ; next; ) +]) + +AT_CHECK([grep "lr_out_snat" lr0flows | sort], [0], [dnl + table=2 (lr_out_snat ), priority=0 , match=(1), action=(next;) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-public"), action=(ct_snat(def0::10);) + table=2 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw0"), action=(ct_snat(aef0::1);) + table=2 (lr_out_snat ), priority=120 , match=(nd_ns), action=(next;) + table=2 (lr_out_snat ), priority=25 , match=(ip && ip4.src == 10.0.0.0/24), action=(ct_snat(172.168.0.10);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.10), action=(ct_snat(172.168.0.30);) + table=2 (lr_out_snat ), priority=33 , match=(ip && ip4.src == 10.0.0.3), action=(ct_snat(172.168.0.20);) +]) + +AT_CLEANUP +]) diff --git a/tests/ovn.at b/tests/ovn.at index bc494fcad9bb..ea1593197f21 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -20571,7 +20571,7 @@ AT_CAPTURE_FILE([sbflows2]) OVS_WAIT_FOR_OUTPUT( [ovn-sbctl dump-flows > sbflows2 ovn-sbctl dump-flows lr0 | grep ct_lb | grep priority=120 | sed 's/table=..//'], 0, - [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80,20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) + [ (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_lb(backends=10.0.0.3:80,20.0.0.3:80; hash_fields="ip_dst,ip_src,tcp_dst,tcp_src");) ]) # get the svc monitor mac. @@ -20612,8 +20612,8 @@ AT_CHECK( AT_CAPTURE_FILE([sbflows4]) ovn-sbctl dump-flows lr0 > sbflows4 AT_CHECK([grep lr_in_dnat sbflows4 | grep priority=120 | sed 's/table=..//' | sort], [0], [dnl - (lr_in_dnat ), priority=120 , match=(ct.est && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;) - (lr_in_dnat ), priority=120 , match=(ct.new && ip && ip4.dst == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(drop;) + (lr_in_dnat ), priority=120 , match=(ct.est && ip && reg0 == 10.0.0.10 && ct_label.natted == 1 && tcp && is_chassis_resident("cr-lr0-public")), action=(next;) + (lr_in_dnat ), priority=120 , match=(ct.new && ip && reg0 == 10.0.0.10 && tcp && tcp.dst == 80 && is_chassis_resident("cr-lr0-public")), action=(drop;) ]) # Delete sw0-p1 diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 552fdae52665..4f104171bdba 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -116,6 +116,7 @@ NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 | FORMAT_PING], \ # Check conntrack entries. AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.2) | \ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +icmp,orig=(src=172.16.1.2,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> icmp,orig=(src=172.16.1.2,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared> ]) @@ -298,6 +299,7 @@ NS_CHECK_EXEC([alice1], [ping6 -q -c 3 -i 0.3 -w 2 fd30::2 | FORMAT_PING], \ # Check conntrack entries. AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(fd21::2) | \ sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl +icmpv6,orig=(src=fd21::2,dst=fd11::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> icmpv6,orig=(src=fd21::2,dst=fd30::2,id=<cleared>,type=128,code=0),reply=(src=fd11::2,dst=fd21::2,id=<cleared>,type=129,code=0),zone=<cleared> ]) @@ -2197,11 +2199,12 @@ tcp,orig=(src=172.16.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(sr ]) check_est_flows () { - n=$(ovs-ofctl dump-flows br-int table=15 | grep \ -"priority=120,ct_state=+est+trk,tcp,metadata=0x2,nw_dst=30.0.0.2,tp_dst=8000" \ -| grep nat | sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') + n=$(ovs-ofctl dump-flows br-int table=13 | grep \ +"priority=100,tcp,metadata=0x2,nw_dst=30.0.0.2" | grep nat | +sed -n 's/.*n_packets=\([[0-9]]\{1,\}\).*/\1/p') echo "n_packets=$n" + test ! -z $n test "$n" != 0 } @@ -2222,7 +2225,7 @@ ovn-nbctl set load_balancer $uuid vips:'"30.0.0.2:8000"'='"192.168.1.2:80,192.16 ovn-nbctl list load_balancer ovn-sbctl dump-flows R2 -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ grep 'nat(src=20.0.0.2)']) dnl Test load-balancing that includes L4 ports in NAT. @@ -2260,7 +2263,7 @@ ovn-nbctl set load_balancer $uuid vips:'"30.0.0.2:8000"'='"192.168.1.2:80,192.16 ovn-nbctl list load_balancer ovn-sbctl dump-flows R2 -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=41 | \ +OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-flows br-int table=42 | \ grep 'nat(src=20.0.0.2)']) rm -f wget*.log