| Message ID | 20250513104959.326659-1-vsaienko@mirantis.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [ovs-dev,v2] Propagate MAC database from VTEP endpoints. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | fail | github build: failed |
| ovsrobot/github-robot-_ovn-kubernetes | fail | github build: failed |
Hello I'm trying to implement the following TODO* Update learned MAC addresses from VTEP to OVN *from [0]. I'm trying to push macs from Mcast_Macs_Local to OVN SB Mac_Binding. In Mcast_Macs_Local ip address is optional while in Mac_Binding it seems empty string is allowed but services will complain for such mac with the wolling error: +2025-05-13T11:07:35.485Z|00048|lflow|WARN|bad 'ip' +2025-05-13T11:07:35.598Z|00051|mac_cache|WARN|Couldn't parse MAC binding: ip=, mac=02:00:00:00:00:ff So my question is Mac_Binding is the correct table which we can use for VTEP mac synchronization? [0] Update learned MAC addresses from VTEP to OVN On Tue, May 13, 2025 at 1:50 PM Vasyl Saienko <vsaienko@mirantis.com> wrote: > With this patch ovn-controller-vtep start synchronizing local > MAC database to Mac_Binding OVN SB table. Which helps to resolve > couple of problems: > > 1. Connectivity between hosts plugged to different switches. > Currently OVN does not create tunnel between hardware switches > even when they have ports plugged to same network. > With this patch ovn-controller-vtep checking for Mac_Binding > OVN SB database and setup required tunnels between switches. > > 2. OVN rely on broadcasts when send data to remote VTEP MACs. > Since now we propagate switch local MAC table to SB database, > later this information may be used by ovn-controller instances > to create direct rules for hosts discovered from remote VTEPs. > > Signed-off-by: Vasyl Saienko <vsaienko@mirantis.com> > --- > controller-vtep/ovn-controller-vtep.c | 9 + > controller-vtep/vtep.c | 301 +++++++++++++++++++++++++- > tests/ovn-controller-vtep.at | 124 ++++++++++- > 3 files changed, 425 insertions(+), 9 deletions(-) > > diff --git a/controller-vtep/ovn-controller-vtep.c > b/controller-vtep/ovn-controller-vtep.c > index 698511482..7e64bacf5 100644 > --- a/controller-vtep/ovn-controller-vtep.c > +++ b/controller-vtep/ovn-controller-vtep.c > @@ -172,6 +172,15 @@ main(int argc, char *argv[]) > ovsdb_idl_add_column(ovnsb_idl_loop.idl, > &sbrec_port_binding_col_type); > ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_up); > > + /* Listen for Mac_Binding changes, we use this table to create tunnels > + * for macs learned from other VTEPs */ > + ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_mac_binding); > + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_mac); > + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_ip); > + ovsdb_idl_add_column(ovnsb_idl_loop.idl, > + &sbrec_mac_binding_col_logical_port); > + ovsdb_idl_add_column(ovnsb_idl_loop.idl, > &sbrec_mac_binding_col_datapath); > + > ovsdb_idl_set_leader_only(ovnsb_idl_loop.idl, false); > ovsdb_idl_get_initial_snapshot(ovnsb_idl_loop.idl); > > diff --git a/controller-vtep/vtep.c b/controller-vtep/vtep.c > index 7f5e1d606..0975b11f4 100644 > --- a/controller-vtep/vtep.c > +++ b/controller-vtep/vtep.c > @@ -42,6 +42,11 @@ struct mmr_hash_node_data { > struct shash physical_locators; > }; > > +struct lp_mac_ip_binding { > + const char *lp; > + struct shash mac_ip_lp; > +}; > + > /* > * Scans through the Binding table in ovnsb, and updates the vtep logical > * switch tunnel keys and the 'Ucast_Macs_Remote' table in the VTEP > @@ -90,7 +95,8 @@ create_pl(struct ovsdb_idl_txn *vtep_idl_txn, const char > *chassis_ip) > vteprec_physical_locator_insert(vtep_idl_txn); > > vteprec_physical_locator_set_dst_ip(new_pl, chassis_ip); > - vteprec_physical_locator_set_encapsulation_type(new_pl, > VTEP_ENCAP_TYPE); > + vteprec_physical_locator_set_encapsulation_type(new_pl, > + VTEP_ENCAP_TYPE); > > return new_pl; > } > @@ -267,9 +273,15 @@ vtep_lswitch_run(struct shash *vtep_pbs, struct sset > *vtep_pswitches, > /* Updates the vtep 'Ucast_Macs_Remote' and 'Mcast_Macs_Remote' tables > based > * on non-vtep port bindings. */ > static void > -vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash > *ucast_macs_rmts, > - struct shash *mcast_macs_rmts, struct shash > *physical_locators, > - struct shash *vtep_lswitches, struct shash *non_vtep_pbs) > +vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, > + struct shash *ucast_macs_rmts, > + struct shash *mcast_macs_rmts, > + struct shash *physical_locators, > + struct shash *vtep_lswitches, > + struct shash *non_vtep_pbs, > + struct shash *vtep_pbs, > + struct shash *sbrec_lp_mac_binding, > + struct sset *vtep_pswitches) > { > struct shash_node *node; > struct hmap ls_map; > @@ -344,6 +356,14 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, > struct shash *ucast_macs_rmts, > continue; > } > tnl_key = peer_pb->datapath->tunnel_key; > + } else if (!strcmp(port_binding_rec->type, "external")) { > + /* External ports sits actually behind remote VTEPs > + * but port itself is bound to one of gateway nodes > + * to provide DHCP/Metadata. Skip port_binding > + * information for such ports, as its does not specify > + * real node location. Use dynamically learned Mac_Binding > + * records from remote VTEPs */ > + continue; > } else { > tnl_key = port_binding_rec->datapath->tunnel_key; > } > @@ -445,6 +465,118 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, > struct shash *ucast_macs_rmts, > } > } > > + /* Handle dynamically leart MACs from remote VTEPs registered in > + * Mac_Binding table. */ > + SHASH_FOR_EACH (node, vtep_pbs) { > + const struct sbrec_port_binding *port_binding_rec = node->data; > + const struct sbrec_chassis *chassis_rec; > + struct ls_hash_node *ls_node; > + const char *chassis_ip; > + int64_t tnl_key; > + > + chassis_rec = port_binding_rec->chassis; > + if (!chassis_rec) { > + continue; > + } > + > + const char *pswitch_name = smap_get(&port_binding_rec->options, > + "vtep-physical-switch"); > + /* Ignore macs learned by ourselfs */ > + if (sset_find(vtep_pswitches, pswitch_name)) { > + continue; > + } > + tnl_key = port_binding_rec->datapath->tunnel_key; > + > + HMAP_FOR_EACH_WITH_HASH (ls_node, hmap_node, > + hash_uint64((uint64_t) tnl_key), > + &ls_map) { > + if (ls_node->vtep_ls->tunnel_key[0] == tnl_key) { > + break; > + } > + } > + /* If 'ls_node' is NULL, that means no vtep logical switch is > + * attached to the corresponding ovn logical datapath, so pass. > + */ > + if (!ls_node) { > + continue; > + } > + > + chassis_ip = get_chassis_vtep_ip(chassis_rec); > + /* Unreachable chassis, continue. */ > + if (!chassis_ip) { > + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); > + VLOG_INFO_RL(&rl, "VTEP tunnel encap on chassis (%s) not > found", > + chassis_rec->name); > + continue; > + } > + > + const struct vteprec_physical_locator *pl = > + shash_find_data(physical_locators, chassis_ip); > + if (!pl) { > + pl = create_pl(vtep_idl_txn, chassis_ip); > + shash_add(physical_locators, chassis_ip, pl); > + } > + > + const struct vteprec_physical_locator *ls_pl = > + shash_find_data(&ls_node->physical_locators, chassis_ip); > + if (!ls_pl) { > + struct vtep_rec_physical_locator_list_entry *ploc_entry = > + xmalloc(sizeof *ploc_entry); > + ploc_entry->vteprec_ploc = pl; > + ovs_list_push_back(&ls_node->locators_list, > + &ploc_entry->locators_node); > + shash_add(&ls_node->physical_locators, chassis_ip, pl); > + } > + > + > + struct lp_mac_ip_binding *miplpb = shash_find_data( > + sbrec_lp_mac_binding, port_binding_rec->logical_port); > + struct shash_node *miplb_node; > + if (!miplpb) { > + continue; > + } > + SHASH_FOR_EACH (miplb_node, &miplpb->mac_ip_lp) { > + const struct sbrec_mac_binding * mb = miplb_node->data; > + /* Ignore MACs from other networks (datapathese) */ > + if (port_binding_rec->datapath->tunnel_key != > + mb->datapath->tunnel_key) { > + continue; > + } > + const struct vteprec_ucast_macs_remote *umr; > + const struct sbrec_port_binding *conflict; > + > + char *mac = mb->mac; > + > + /* Checks for duplicate MAC in the same vtep logical switch. > */ > + conflict = shash_find_data(&ls_node->added_macs, mac); > + if (conflict) { > + VLOG_WARN("MAC address (%s) has already been known to be " > + "on logical port (%s) in the same logical " > + "datapath, so just ignore this logical port > (%s)", > + mac, conflict->logical_port, > + port_binding_rec->logical_port); > + continue; > + } > + shash_add(&ls_node->added_macs, mac, port_binding_rec); > + > + char *mac_ip_tnlkey = xasprintf("%s_%s_%"PRId64, mac, > chassis_ip, > + tnl_key); > + umr = shash_find_data(ucast_macs_rmts, mac_ip_tnlkey); > + /* If finds the 'umr' entry for the mac, ip, and tnl_key, > deletes > + * the entry from shash so that it is not garbage collected. > + * > + * If not found, creates a new 'umr' entry. */ > + if (umr && umr->logical_switch == ls_node->vtep_ls) { > + shash_find_and_delete(ucast_macs_rmts, mac_ip_tnlkey); > + } else { > + const struct vteprec_ucast_macs_remote *new_umr; > + new_umr = create_umr(vtep_idl_txn, mac, ls_node->vtep_ls); > + vteprec_ucast_macs_remote_set_locator(new_umr, pl); > + } > + free(mac_ip_tnlkey); > + } > + } > + > /* Removes all remaining 'umr's, since they do not exist anymore. */ > SHASH_FOR_EACH (node, ucast_macs_rmts) { > vteprec_ucast_macs_remote_delete(node->data); > @@ -454,7 +586,7 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, > struct shash *ucast_macs_rmts, > struct vtep_rec_physical_locator_list_entry *ploc_entry; > vtep_update_mmr(vtep_idl_txn, &iter->locators_list, > iter->vtep_ls, iter->mmr_ext); > - LIST_FOR_EACH_POP(ploc_entry, locators_node, > + LIST_FOR_EACH_POP (ploc_entry, locators_node, > &iter->locators_list) { > free(ploc_entry); > } > @@ -519,6 +651,123 @@ vtep_mcast_macs_cleanup(struct ovsdb_idl *vtep_idl) > > return true; > } > + > +static const struct sbrec_port_binding * > +find_pbs_for_logical_switch(struct shash *vtep_pbs, > + struct sset *vtep_pswitches, > + char *ls_name){ > + > + struct shash_node *node; > + SHASH_FOR_EACH (node, vtep_pbs) { > + const struct sbrec_port_binding *port_binding_rec = node->data; > + const char *pswitch_name = smap_get(&port_binding_rec->options, > + "vtep-physical-switch"); > + const char *lswitch_name = smap_get(&port_binding_rec->options, > + "vtep-logical-switch"); > + if (!port_binding_rec->chassis) { > + continue; > + } > + > + /* If 'port_binding_rec->chassis' exists then 'pswitch_name' > + * and 'lswitch_name' must also exist. */ > + if (!pswitch_name || !lswitch_name) { > + /* This could only happen when someone directly modifies the > + * database, (e.g. using ovn-sbctl). */ > + VLOG_ERR("logical port (%s) with no 'options:vtep-physical-" > + "switch' or 'options:vtep-logical-switch' specified " > + "is bound to chassis (%s).", > + port_binding_rec->logical_port, > + port_binding_rec->chassis->name); > + continue; > + } > + /* Make sure both logical_switch and physical_switch matches */ > + if (!strcmp(ls_name, lswitch_name)) { > + if (sset_find(vtep_pswitches, port_binding_rec->chassis->name)) > { > + return port_binding_rec; > + } > + } > + } > + return NULL; > +} > + > +/* Propagate dynamically learned MACs on local VTEPs to OVN SB. Where > + * later this information is used to create tunnels between neighbour > + * VTEPs. > +*/ > +static void > +vtep_local_macs(struct controller_vtep_ctx *ctx, > + struct shash *vtep_pbs, > + struct sset *vtep_pswitches, > + struct shash *vtep_lswitches, > + struct shash *sbrec_lp_mac_binding){ > + > + if (!ctx->ovnsb_idl_txn) { > + return; > + } > + > + ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn, > + "ovn-controller-vtep: updating > mac_binding"); > + > + const struct vteprec_ucast_macs_local *vtep_uml; > + const struct sbrec_port_binding *port_binding_rec; > + const struct sbrec_mac_binding *mb; > + > + /* Collect local unicast MACs */ > + VTEPREC_UCAST_MACS_LOCAL_FOR_EACH (vtep_uml, ctx->vtep_idl) { > + port_binding_rec = find_pbs_for_logical_switch( > + vtep_pbs, vtep_pswitches, vtep_uml->logical_switch->name); > + if (!port_binding_rec) { > + VLOG_ERR("Cannot find port_binding for dynamically learned > MAC %s " > + "in logical_switch %s", vtep_uml->MAC, > + vtep_uml->logical_switch->name); > + continue; > + } > + > + struct lp_mac_ip_binding *miplpb = shash_find_data( > + sbrec_lp_mac_binding, port_binding_rec->logical_port); > + mb = NULL; > + if (miplpb) { > + char *mac_ip_lp_key = xasprintf("%s_%s_%s", vtep_uml->MAC, > + vtep_uml->ipaddr, > + > port_binding_rec->logical_port); > + mb = shash_find_and_delete(&miplpb->mac_ip_lp, mac_ip_lp_key); > + free(mac_ip_lp_key); > + } > + > + if (!mb) { > + VLOG_DBG("Creating new mac_binding entry for mac %s", > + vtep_uml->MAC); > + mb = sbrec_mac_binding_insert(ctx->ovnsb_idl_txn); > + sbrec_mac_binding_set_mac(mb, vtep_uml->MAC); > + sbrec_mac_binding_set_logical_port( > + mb, port_binding_rec->logical_port); > + sbrec_mac_binding_set_timestamp(mb, time_wall_msec()); > + } > + sbrec_mac_binding_set_ip(mb, vtep_uml->ipaddr); > + sbrec_mac_binding_set_datapath(mb, port_binding_rec->datapath); > + } > + > + struct shash_node *node; > + struct shash_node *miplb_node; > + SHASH_FOR_EACH (node, vtep_lswitches) { > + port_binding_rec = find_pbs_for_logical_switch(vtep_pbs, > + vtep_pswitches, > + node->name); > + if (!port_binding_rec) { > + continue; > + } > + struct lp_mac_ip_binding *miplpb = shash_find_data( > + sbrec_lp_mac_binding, port_binding_rec->logical_port); > + if (!miplpb) { > + continue; > + } > + SHASH_FOR_EACH (miplb_node, &miplpb->mac_ip_lp) { > + mb = miplb_node->data; > + VLOG_DBG("Removing mac_binding for stale VTEP mac %s", > mb->mac); > + sbrec_mac_binding_delete(mb); > + } > + } > +} > > /* Updates vtep logical switch tunnel keys. */ > void > @@ -593,7 +842,7 @@ vtep_run(struct controller_vtep_ctx *ctx) > } > > /* Collects and classifies 'Port_Binding's. */ > - SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->ovnsb_idl) { > + SBREC_PORT_BINDING_FOR_EACH (port_binding_rec, ctx->ovnsb_idl) { > struct shash *target = > !strcmp(port_binding_rec->type, "vtep") ? &vtep_pbs > : &non_vtep_pbs; > @@ -605,6 +854,33 @@ vtep_run(struct controller_vtep_ctx *ctx) > shash_add(target, port_binding_rec->logical_port, > port_binding_rec); > } > > + /* Construct logical_port to mac_binding */ > + const struct sbrec_mac_binding *mb; > + struct shash sbrec_lp_mac_binding = SHASH_INITIALIZER( > + &sbrec_lp_mac_binding); > + SBREC_MAC_BINDING_FOR_EACH (mb, ctx->ovnsb_idl) { > + if (!mb->logical_port) { > + continue; > + } > + char *mac_ip_lp_key = xasprintf("%s_%s_%s", mb->mac, mb->ip, > + mb->logical_port); > + > + struct lp_mac_ip_binding *miplpb = shash_find_data( > + &sbrec_lp_mac_binding, mb->logical_port); > + if (!miplpb) { > + struct lp_mac_ip_binding *sbrec_mac_ip_lp_binding = xmalloc( > + sizeof *sbrec_mac_ip_lp_binding); > + shash_init(&sbrec_mac_ip_lp_binding->mac_ip_lp); > + sbrec_mac_ip_lp_binding->lp = mb->logical_port; > + shash_add(&sbrec_mac_ip_lp_binding->mac_ip_lp, mac_ip_lp_key, > mb); > + shash_add(&sbrec_lp_mac_binding, mb->logical_port, > + sbrec_mac_ip_lp_binding); > + } else { > + shash_add(&miplpb->mac_ip_lp, mac_ip_lp_key, mb); > + } > + free(mac_ip_lp_key); > + } > + > ovsdb_idl_txn_add_comment(ctx->vtep_idl_txn, > "ovn-controller-vtep: update logical switch > " > "tunnel keys and 'ucast_macs_remote's"); > @@ -612,7 +888,10 @@ vtep_run(struct controller_vtep_ctx *ctx) > vtep_lswitch_run(&vtep_pbs, &vtep_pswitches, &vtep_lswitches); > vtep_macs_run(ctx->vtep_idl_txn, &ucast_macs_rmts, > &mcast_macs_rmts, &physical_locators, > - &vtep_lswitches, &non_vtep_pbs); > + &vtep_lswitches, &non_vtep_pbs, &vtep_pbs, > + &sbrec_lp_mac_binding, &vtep_pswitches); > + vtep_local_macs(ctx, &vtep_pbs, &vtep_pswitches, &vtep_lswitches, > + &sbrec_lp_mac_binding); > > sset_destroy(&vtep_pswitches); > shash_destroy(&vtep_lswitches); > @@ -628,6 +907,14 @@ vtep_run(struct controller_vtep_ctx *ctx) > shash_destroy(&physical_locators); > shash_destroy(&vtep_pbs); > shash_destroy(&non_vtep_pbs); > + > + struct shash_node *lp_mb_node; > + SHASH_FOR_EACH (lp_mb_node, &sbrec_lp_mac_binding) { > + struct lp_mac_ip_binding *miplpb = lp_mb_node->data; > + shash_destroy(&miplpb->mac_ip_lp); > + free(miplpb); > + } > + shash_destroy(&sbrec_lp_mac_binding); > } > > /* Cleans up all related entries in vtep. Returns true when done (i.e. > there > diff --git a/tests/ovn-controller-vtep.at b/tests/ovn-controller-vtep.at > index d410ecd28..a0a4b89ff 100644 > --- a/tests/ovn-controller-vtep.at > +++ b/tests/ovn-controller-vtep.at > @@ -1,9 +1,10 @@ > AT_BANNER([ovn_controller_vtep]) > > -# OVN_CONTROLLER_VTEP_START(SIM_NAME) > +# OVN_CONTROLLER_VTEP_START(SIM_NAME, TUNNEL_IP) > # > # $1 - optional simulator name. If none is given, runs > ovn-controller-vtep, and > # vtep emulator in $ovs_dir. > +# $2 - optional tunnel ip. If none is given default 1.2.3.4 is used > # Starts the test with a setup with vtep device. Each test case must > first > # call this macro and ovn_start. > # > @@ -14,7 +15,9 @@ m4_define([OVN_CONTROLLER_VTEP_START], [ > AT_KEYWORDS([ovn]) > # this will cause skip when 'make check' using Windows setup. > sim="$1" > + tunnel_ip="$2" > vtep_chassis=${sim:-br-vtep} > + tunnel_ip=${tunnel_ip:-1.2.3.4} > > test -n "$sim" && as "$sim" > mkdir -p "$ovs_dir" || return 1 > @@ -41,7 +44,7 @@ m4_define([OVN_CONTROLLER_VTEP_START], [ > -- add-port $vtep_chassis p1 -- set Interface p1 type=dummy > ofport_request=2 > > dnl Start ovs-vtep. > - check vtep-ctl add-ps $vtep_chassis -- set Physical_Switch > $vtep_chassis tunnel_ips=1.2.3.4 > + check vtep-ctl add-ps $vtep_chassis -- set Physical_Switch > $vtep_chassis tunnel_ips=$tunnel_ip > AT_CHECK([ovs-vtep --log-file="$ovs_dir"/ovs-vtep.log \ > --pidfile="$ovs_dir"/ovs-vtep.pid \ > --detach --no-chdir $vtep_chassis], [0], [], > [stderr]) > @@ -86,6 +89,9 @@ AT_CHECK([ovn-nbctl lsp-set-type $2 vtep]) > AT_CHECK([ovn-nbctl lsp-set-options $2 vtep-physical-switch=$3 > vtep-logical-switch=$4]) > ]) > > +# mac learning table is hardcoded in ovs-vtep emulator > +m4_define([mac_learning_table], [1]) > + > ############################################## > > # tests chassis related updates. > @@ -509,6 +515,120 @@ AT_CHECK([vtep-ctl --columns=MAC list > Ucast_Macs_Remote | cut -d ':' -f2- | tr - > OVN_CONTROLLER_VTEP_STOP([/has already been known to be on logical > port/d]) > AT_CLEANUP > > +# Tests vtep module 'Ucast_Macs_Local' MACs propagation > +AT_SETUP([ovn-controller-vtep - vtep-macs 3]) > +ovn_start > +OVN_CONTROLLER_VTEP_START > + > +check ovn-nbctl ls-add br-test > +check ovn-nbctl ls-add br-test1 > + > +# creates a simple logical network with the vtep device and a fake hv > chassis > +# 'ch0'. > +AT_CHECK([ovn-nbctl --wait=sb sync]) > + > +# creates the logical switch in vtep and adds the corresponding logical > +# port to 'br-test'. > +AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0]) > +OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0]) > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep > br-vtep_lswitch0`"]) > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep_vtep_ls1`"]) > + > +# Simulate peer on physical port to populate Ucast_Macs_Local table > +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep_vtep_ls1 > "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ee:01 > actions=output:0100-p0-l"]) > + > +# Wait Ucast_Macs_Local is updated with local MAC > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep > aa:bb:cc:dd:ee:01`"]) > + > +# Check that OVNSB is updated with learned MAC > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl find Mac_Binding > mac='"aa:bb:cc:dd:ee:01"' |grep _uuid`"]) > + > +# Plug p1 to lswitch1 > +AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep p1 100 lswitch1]) > +OVN_NB_ADD_VTEP_PORT([br-test1], [br-vtep_lswitch1], [br-vtep], > [lswitch1]) > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep > br-vtep_lswitch1`"]) > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep_vtep_ls2`"]) > + > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p1-l`"]) > + > +# Simulate peer on physical port to populate Ucast_Macs_Local table > +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep_vtep_ls2 > "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ee:02 > actions=output:0100-p1-l"]) > + > +# Wait Ucast_Macs_Local is updated with local MAC > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep > aa:bb:cc:dd:ee:02`"]) > + > +# Remove MAC from Ucast_Macs_Local > +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep_vtep_ls1 > "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ee:01"]) > + > +# Check that first MAC is removed from OVNSB, but second is still there > +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding > mac='"aa:bb:cc:dd:ee:01"'`"]) > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl find Mac_Binding > mac='"aa:bb:cc:dd:ee:02"'`"]) > + > +# Remove second MAC from Ucast_Macs_Local > +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep_vtep_ls2 > "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ee:02"]) > + > +# Check that second MAC is removed from OVNSB > +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding > mac='"aa:bb:cc:dd:ee:01"'`"]) > +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding > mac='"aa:bb:cc:dd:ee:02"'`"]) > + > +# Start second VTEP chassis > +OVN_CONTROLLER_VTEP_START(br-vtep1, 1.2.3.5) > + > +# creates the logical switch in vtep and adds the corresponding logical > +# # port to 'br-test'. > +AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep1 p0 100 lswitch0]) > +OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep1_lswitch0], [br-vtep1], > [lswitch0]) > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep > br-vtep1_lswitch0`"]) > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep1_vtep_ls1`"]) > + > +# creates the logical switch in vtep and adds the corresponding logical > +# # # port to 'br-test'. > +AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep1 p1 100 lswitch1]) > +OVN_NB_ADD_VTEP_PORT([br-test1], [br-vtep1_lswitch1], [br-vtep1], > [lswitch1]) > +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep > br-vtep1_lswitch1`"]) > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep1_vtep_ls2`"]) > + > +#sleep 3000 > + > +# Wait bingins are activated > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p0-l`"]) > +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p1-l`"]) > + > +# Simulate peer on physical port to populate Ucast_Macs_Local table on > br-vtep1 > +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep1_vtep_ls1 > "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ff:01 > actions=output:0100-p0-l"]) > +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep1_vtep_ls2 > "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ff:02 > actions=output:0100-p1-l"]) > + > +# Wait Ucast_Macs_Local is updated with local MAC > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep > aa:bb:cc:dd:ff:01`"]) > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep > aa:bb:cc:dd:ff:02`"]) > + > +# Switch context > +as > + > +# Make sure we see > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list-remote-macs lswitch0 | grep > aa:bb:cc:dd:ff:01`"]) > +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch1 | grep > aa:bb:cc:dd:ff:01`"]) > + > +OVS_WAIT_UNTIL([test -n "`vtep-ctl list-remote-macs lswitch1 | grep > aa:bb:cc:dd:ff:02`"]) > +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch0 | grep > aa:bb:cc:dd:ff:02`"]) > + > +# Switch context to br-vtep1 > +as br-vtep1 > + > +# Remove local MACs > +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep1_vtep_ls1 > "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ff:01"]) > +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep1_vtep_ls2 > "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ff:02"]) > + > +# Switch context > +as > + > +# Ensure remote MACs gone > +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch0 | grep > aa:bb:cc:dd:ff:01`"]) > +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch1 | grep > aa:bb:cc:dd:ff:02`"]) > + > +OVN_CONTROLLER_VTEP_STOP > +AT_CLEANUP > + > # Tests vtep module 'Mcast_Macs_Remote's. > AT_SETUP([ovn-controller-vtep - vtep-Mcast_Macs_Remote]) > ovn_start > -- > 2.43.0 > >
diff --git a/controller-vtep/ovn-controller-vtep.c b/controller-vtep/ovn-controller-vtep.c index 698511482..7e64bacf5 100644 --- a/controller-vtep/ovn-controller-vtep.c +++ b/controller-vtep/ovn-controller-vtep.c @@ -172,6 +172,15 @@ main(int argc, char *argv[]) ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_type); ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_port_binding_col_up); + /* Listen for Mac_Binding changes, we use this table to create tunnels + * for macs learned from other VTEPs */ + ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_mac_binding); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_mac); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_ip); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, + &sbrec_mac_binding_col_logical_port); + ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_datapath); + ovsdb_idl_set_leader_only(ovnsb_idl_loop.idl, false); ovsdb_idl_get_initial_snapshot(ovnsb_idl_loop.idl); diff --git a/controller-vtep/vtep.c b/controller-vtep/vtep.c index 7f5e1d606..0975b11f4 100644 --- a/controller-vtep/vtep.c +++ b/controller-vtep/vtep.c @@ -42,6 +42,11 @@ struct mmr_hash_node_data { struct shash physical_locators; }; +struct lp_mac_ip_binding { + const char *lp; + struct shash mac_ip_lp; +}; + /* * Scans through the Binding table in ovnsb, and updates the vtep logical * switch tunnel keys and the 'Ucast_Macs_Remote' table in the VTEP @@ -90,7 +95,8 @@ create_pl(struct ovsdb_idl_txn *vtep_idl_txn, const char *chassis_ip) vteprec_physical_locator_insert(vtep_idl_txn); vteprec_physical_locator_set_dst_ip(new_pl, chassis_ip); - vteprec_physical_locator_set_encapsulation_type(new_pl, VTEP_ENCAP_TYPE); + vteprec_physical_locator_set_encapsulation_type(new_pl, + VTEP_ENCAP_TYPE); return new_pl; } @@ -267,9 +273,15 @@ vtep_lswitch_run(struct shash *vtep_pbs, struct sset *vtep_pswitches, /* Updates the vtep 'Ucast_Macs_Remote' and 'Mcast_Macs_Remote' tables based * on non-vtep port bindings. */ static void -vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash *ucast_macs_rmts, - struct shash *mcast_macs_rmts, struct shash *physical_locators, - struct shash *vtep_lswitches, struct shash *non_vtep_pbs) +vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, + struct shash *ucast_macs_rmts, + struct shash *mcast_macs_rmts, + struct shash *physical_locators, + struct shash *vtep_lswitches, + struct shash *non_vtep_pbs, + struct shash *vtep_pbs, + struct shash *sbrec_lp_mac_binding, + struct sset *vtep_pswitches) { struct shash_node *node; struct hmap ls_map; @@ -344,6 +356,14 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash *ucast_macs_rmts, continue; } tnl_key = peer_pb->datapath->tunnel_key; + } else if (!strcmp(port_binding_rec->type, "external")) { + /* External ports sits actually behind remote VTEPs + * but port itself is bound to one of gateway nodes + * to provide DHCP/Metadata. Skip port_binding + * information for such ports, as its does not specify + * real node location. Use dynamically learned Mac_Binding + * records from remote VTEPs */ + continue; } else { tnl_key = port_binding_rec->datapath->tunnel_key; } @@ -445,6 +465,118 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash *ucast_macs_rmts, } } + /* Handle dynamically leart MACs from remote VTEPs registered in + * Mac_Binding table. */ + SHASH_FOR_EACH (node, vtep_pbs) { + const struct sbrec_port_binding *port_binding_rec = node->data; + const struct sbrec_chassis *chassis_rec; + struct ls_hash_node *ls_node; + const char *chassis_ip; + int64_t tnl_key; + + chassis_rec = port_binding_rec->chassis; + if (!chassis_rec) { + continue; + } + + const char *pswitch_name = smap_get(&port_binding_rec->options, + "vtep-physical-switch"); + /* Ignore macs learned by ourselfs */ + if (sset_find(vtep_pswitches, pswitch_name)) { + continue; + } + tnl_key = port_binding_rec->datapath->tunnel_key; + + HMAP_FOR_EACH_WITH_HASH (ls_node, hmap_node, + hash_uint64((uint64_t) tnl_key), + &ls_map) { + if (ls_node->vtep_ls->tunnel_key[0] == tnl_key) { + break; + } + } + /* If 'ls_node' is NULL, that means no vtep logical switch is + * attached to the corresponding ovn logical datapath, so pass. + */ + if (!ls_node) { + continue; + } + + chassis_ip = get_chassis_vtep_ip(chassis_rec); + /* Unreachable chassis, continue. */ + if (!chassis_ip) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); + VLOG_INFO_RL(&rl, "VTEP tunnel encap on chassis (%s) not found", + chassis_rec->name); + continue; + } + + const struct vteprec_physical_locator *pl = + shash_find_data(physical_locators, chassis_ip); + if (!pl) { + pl = create_pl(vtep_idl_txn, chassis_ip); + shash_add(physical_locators, chassis_ip, pl); + } + + const struct vteprec_physical_locator *ls_pl = + shash_find_data(&ls_node->physical_locators, chassis_ip); + if (!ls_pl) { + struct vtep_rec_physical_locator_list_entry *ploc_entry = + xmalloc(sizeof *ploc_entry); + ploc_entry->vteprec_ploc = pl; + ovs_list_push_back(&ls_node->locators_list, + &ploc_entry->locators_node); + shash_add(&ls_node->physical_locators, chassis_ip, pl); + } + + + struct lp_mac_ip_binding *miplpb = shash_find_data( + sbrec_lp_mac_binding, port_binding_rec->logical_port); + struct shash_node *miplb_node; + if (!miplpb) { + continue; + } + SHASH_FOR_EACH (miplb_node, &miplpb->mac_ip_lp) { + const struct sbrec_mac_binding * mb = miplb_node->data; + /* Ignore MACs from other networks (datapathese) */ + if (port_binding_rec->datapath->tunnel_key != + mb->datapath->tunnel_key) { + continue; + } + const struct vteprec_ucast_macs_remote *umr; + const struct sbrec_port_binding *conflict; + + char *mac = mb->mac; + + /* Checks for duplicate MAC in the same vtep logical switch. */ + conflict = shash_find_data(&ls_node->added_macs, mac); + if (conflict) { + VLOG_WARN("MAC address (%s) has already been known to be " + "on logical port (%s) in the same logical " + "datapath, so just ignore this logical port (%s)", + mac, conflict->logical_port, + port_binding_rec->logical_port); + continue; + } + shash_add(&ls_node->added_macs, mac, port_binding_rec); + + char *mac_ip_tnlkey = xasprintf("%s_%s_%"PRId64, mac, chassis_ip, + tnl_key); + umr = shash_find_data(ucast_macs_rmts, mac_ip_tnlkey); + /* If finds the 'umr' entry for the mac, ip, and tnl_key, deletes + * the entry from shash so that it is not garbage collected. + * + * If not found, creates a new 'umr' entry. */ + if (umr && umr->logical_switch == ls_node->vtep_ls) { + shash_find_and_delete(ucast_macs_rmts, mac_ip_tnlkey); + } else { + const struct vteprec_ucast_macs_remote *new_umr; + new_umr = create_umr(vtep_idl_txn, mac, ls_node->vtep_ls); + vteprec_ucast_macs_remote_set_locator(new_umr, pl); + } + free(mac_ip_tnlkey); + } + } + /* Removes all remaining 'umr's, since they do not exist anymore. */ SHASH_FOR_EACH (node, ucast_macs_rmts) { vteprec_ucast_macs_remote_delete(node->data); @@ -454,7 +586,7 @@ vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash *ucast_macs_rmts, struct vtep_rec_physical_locator_list_entry *ploc_entry; vtep_update_mmr(vtep_idl_txn, &iter->locators_list, iter->vtep_ls, iter->mmr_ext); - LIST_FOR_EACH_POP(ploc_entry, locators_node, + LIST_FOR_EACH_POP (ploc_entry, locators_node, &iter->locators_list) { free(ploc_entry); } @@ -519,6 +651,123 @@ vtep_mcast_macs_cleanup(struct ovsdb_idl *vtep_idl) return true; } + +static const struct sbrec_port_binding * +find_pbs_for_logical_switch(struct shash *vtep_pbs, + struct sset *vtep_pswitches, + char *ls_name){ + + struct shash_node *node; + SHASH_FOR_EACH (node, vtep_pbs) { + const struct sbrec_port_binding *port_binding_rec = node->data; + const char *pswitch_name = smap_get(&port_binding_rec->options, + "vtep-physical-switch"); + const char *lswitch_name = smap_get(&port_binding_rec->options, + "vtep-logical-switch"); + if (!port_binding_rec->chassis) { + continue; + } + + /* If 'port_binding_rec->chassis' exists then 'pswitch_name' + * and 'lswitch_name' must also exist. */ + if (!pswitch_name || !lswitch_name) { + /* This could only happen when someone directly modifies the + * database, (e.g. using ovn-sbctl). */ + VLOG_ERR("logical port (%s) with no 'options:vtep-physical-" + "switch' or 'options:vtep-logical-switch' specified " + "is bound to chassis (%s).", + port_binding_rec->logical_port, + port_binding_rec->chassis->name); + continue; + } + /* Make sure both logical_switch and physical_switch matches */ + if (!strcmp(ls_name, lswitch_name)) { + if (sset_find(vtep_pswitches, port_binding_rec->chassis->name)) { + return port_binding_rec; + } + } + } + return NULL; +} + +/* Propagate dynamically learned MACs on local VTEPs to OVN SB. Where + * later this information is used to create tunnels between neighbour + * VTEPs. +*/ +static void +vtep_local_macs(struct controller_vtep_ctx *ctx, + struct shash *vtep_pbs, + struct sset *vtep_pswitches, + struct shash *vtep_lswitches, + struct shash *sbrec_lp_mac_binding){ + + if (!ctx->ovnsb_idl_txn) { + return; + } + + ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn, + "ovn-controller-vtep: updating mac_binding"); + + const struct vteprec_ucast_macs_local *vtep_uml; + const struct sbrec_port_binding *port_binding_rec; + const struct sbrec_mac_binding *mb; + + /* Collect local unicast MACs */ + VTEPREC_UCAST_MACS_LOCAL_FOR_EACH (vtep_uml, ctx->vtep_idl) { + port_binding_rec = find_pbs_for_logical_switch( + vtep_pbs, vtep_pswitches, vtep_uml->logical_switch->name); + if (!port_binding_rec) { + VLOG_ERR("Cannot find port_binding for dynamically learned MAC %s " + "in logical_switch %s", vtep_uml->MAC, + vtep_uml->logical_switch->name); + continue; + } + + struct lp_mac_ip_binding *miplpb = shash_find_data( + sbrec_lp_mac_binding, port_binding_rec->logical_port); + mb = NULL; + if (miplpb) { + char *mac_ip_lp_key = xasprintf("%s_%s_%s", vtep_uml->MAC, + vtep_uml->ipaddr, + port_binding_rec->logical_port); + mb = shash_find_and_delete(&miplpb->mac_ip_lp, mac_ip_lp_key); + free(mac_ip_lp_key); + } + + if (!mb) { + VLOG_DBG("Creating new mac_binding entry for mac %s", + vtep_uml->MAC); + mb = sbrec_mac_binding_insert(ctx->ovnsb_idl_txn); + sbrec_mac_binding_set_mac(mb, vtep_uml->MAC); + sbrec_mac_binding_set_logical_port( + mb, port_binding_rec->logical_port); + sbrec_mac_binding_set_timestamp(mb, time_wall_msec()); + } + sbrec_mac_binding_set_ip(mb, vtep_uml->ipaddr); + sbrec_mac_binding_set_datapath(mb, port_binding_rec->datapath); + } + + struct shash_node *node; + struct shash_node *miplb_node; + SHASH_FOR_EACH (node, vtep_lswitches) { + port_binding_rec = find_pbs_for_logical_switch(vtep_pbs, + vtep_pswitches, + node->name); + if (!port_binding_rec) { + continue; + } + struct lp_mac_ip_binding *miplpb = shash_find_data( + sbrec_lp_mac_binding, port_binding_rec->logical_port); + if (!miplpb) { + continue; + } + SHASH_FOR_EACH (miplb_node, &miplpb->mac_ip_lp) { + mb = miplb_node->data; + VLOG_DBG("Removing mac_binding for stale VTEP mac %s", mb->mac); + sbrec_mac_binding_delete(mb); + } + } +} /* Updates vtep logical switch tunnel keys. */ void @@ -593,7 +842,7 @@ vtep_run(struct controller_vtep_ctx *ctx) } /* Collects and classifies 'Port_Binding's. */ - SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->ovnsb_idl) { + SBREC_PORT_BINDING_FOR_EACH (port_binding_rec, ctx->ovnsb_idl) { struct shash *target = !strcmp(port_binding_rec->type, "vtep") ? &vtep_pbs : &non_vtep_pbs; @@ -605,6 +854,33 @@ vtep_run(struct controller_vtep_ctx *ctx) shash_add(target, port_binding_rec->logical_port, port_binding_rec); } + /* Construct logical_port to mac_binding */ + const struct sbrec_mac_binding *mb; + struct shash sbrec_lp_mac_binding = SHASH_INITIALIZER( + &sbrec_lp_mac_binding); + SBREC_MAC_BINDING_FOR_EACH (mb, ctx->ovnsb_idl) { + if (!mb->logical_port) { + continue; + } + char *mac_ip_lp_key = xasprintf("%s_%s_%s", mb->mac, mb->ip, + mb->logical_port); + + struct lp_mac_ip_binding *miplpb = shash_find_data( + &sbrec_lp_mac_binding, mb->logical_port); + if (!miplpb) { + struct lp_mac_ip_binding *sbrec_mac_ip_lp_binding = xmalloc( + sizeof *sbrec_mac_ip_lp_binding); + shash_init(&sbrec_mac_ip_lp_binding->mac_ip_lp); + sbrec_mac_ip_lp_binding->lp = mb->logical_port; + shash_add(&sbrec_mac_ip_lp_binding->mac_ip_lp, mac_ip_lp_key, mb); + shash_add(&sbrec_lp_mac_binding, mb->logical_port, + sbrec_mac_ip_lp_binding); + } else { + shash_add(&miplpb->mac_ip_lp, mac_ip_lp_key, mb); + } + free(mac_ip_lp_key); + } + ovsdb_idl_txn_add_comment(ctx->vtep_idl_txn, "ovn-controller-vtep: update logical switch " "tunnel keys and 'ucast_macs_remote's"); @@ -612,7 +888,10 @@ vtep_run(struct controller_vtep_ctx *ctx) vtep_lswitch_run(&vtep_pbs, &vtep_pswitches, &vtep_lswitches); vtep_macs_run(ctx->vtep_idl_txn, &ucast_macs_rmts, &mcast_macs_rmts, &physical_locators, - &vtep_lswitches, &non_vtep_pbs); + &vtep_lswitches, &non_vtep_pbs, &vtep_pbs, + &sbrec_lp_mac_binding, &vtep_pswitches); + vtep_local_macs(ctx, &vtep_pbs, &vtep_pswitches, &vtep_lswitches, + &sbrec_lp_mac_binding); sset_destroy(&vtep_pswitches); shash_destroy(&vtep_lswitches); @@ -628,6 +907,14 @@ vtep_run(struct controller_vtep_ctx *ctx) shash_destroy(&physical_locators); shash_destroy(&vtep_pbs); shash_destroy(&non_vtep_pbs); + + struct shash_node *lp_mb_node; + SHASH_FOR_EACH (lp_mb_node, &sbrec_lp_mac_binding) { + struct lp_mac_ip_binding *miplpb = lp_mb_node->data; + shash_destroy(&miplpb->mac_ip_lp); + free(miplpb); + } + shash_destroy(&sbrec_lp_mac_binding); } /* Cleans up all related entries in vtep. Returns true when done (i.e. there diff --git a/tests/ovn-controller-vtep.at b/tests/ovn-controller-vtep.at index d410ecd28..a0a4b89ff 100644 --- a/tests/ovn-controller-vtep.at +++ b/tests/ovn-controller-vtep.at @@ -1,9 +1,10 @@ AT_BANNER([ovn_controller_vtep]) -# OVN_CONTROLLER_VTEP_START(SIM_NAME) +# OVN_CONTROLLER_VTEP_START(SIM_NAME, TUNNEL_IP) # # $1 - optional simulator name. If none is given, runs ovn-controller-vtep, and # vtep emulator in $ovs_dir. +# $2 - optional tunnel ip. If none is given default 1.2.3.4 is used # Starts the test with a setup with vtep device. Each test case must first # call this macro and ovn_start. # @@ -14,7 +15,9 @@ m4_define([OVN_CONTROLLER_VTEP_START], [ AT_KEYWORDS([ovn]) # this will cause skip when 'make check' using Windows setup. sim="$1" + tunnel_ip="$2" vtep_chassis=${sim:-br-vtep} + tunnel_ip=${tunnel_ip:-1.2.3.4} test -n "$sim" && as "$sim" mkdir -p "$ovs_dir" || return 1 @@ -41,7 +44,7 @@ m4_define([OVN_CONTROLLER_VTEP_START], [ -- add-port $vtep_chassis p1 -- set Interface p1 type=dummy ofport_request=2 dnl Start ovs-vtep. - check vtep-ctl add-ps $vtep_chassis -- set Physical_Switch $vtep_chassis tunnel_ips=1.2.3.4 + check vtep-ctl add-ps $vtep_chassis -- set Physical_Switch $vtep_chassis tunnel_ips=$tunnel_ip AT_CHECK([ovs-vtep --log-file="$ovs_dir"/ovs-vtep.log \ --pidfile="$ovs_dir"/ovs-vtep.pid \ --detach --no-chdir $vtep_chassis], [0], [], [stderr]) @@ -86,6 +89,9 @@ AT_CHECK([ovn-nbctl lsp-set-type $2 vtep]) AT_CHECK([ovn-nbctl lsp-set-options $2 vtep-physical-switch=$3 vtep-logical-switch=$4]) ]) +# mac learning table is hardcoded in ovs-vtep emulator +m4_define([mac_learning_table], [1]) + ############################################## # tests chassis related updates. @@ -509,6 +515,120 @@ AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2- | tr - OVN_CONTROLLER_VTEP_STOP([/has already been known to be on logical port/d]) AT_CLEANUP +# Tests vtep module 'Ucast_Macs_Local' MACs propagation +AT_SETUP([ovn-controller-vtep - vtep-macs 3]) +ovn_start +OVN_CONTROLLER_VTEP_START + +check ovn-nbctl ls-add br-test +check ovn-nbctl ls-add br-test1 + +# creates a simple logical network with the vtep device and a fake hv chassis +# 'ch0'. +AT_CHECK([ovn-nbctl --wait=sb sync]) + +# creates the logical switch in vtep and adds the corresponding logical +# port to 'br-test'. +AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0]) +OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0]) +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep br-vtep_lswitch0`"]) +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep_vtep_ls1`"]) + +# Simulate peer on physical port to populate Ucast_Macs_Local table +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep_vtep_ls1 "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ee:01 actions=output:0100-p0-l"]) + +# Wait Ucast_Macs_Local is updated with local MAC +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep aa:bb:cc:dd:ee:01`"]) + +# Check that OVNSB is updated with learned MAC +OVS_WAIT_UNTIL([test -n "`ovn-sbctl find Mac_Binding mac='"aa:bb:cc:dd:ee:01"' |grep _uuid`"]) + +# Plug p1 to lswitch1 +AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep p1 100 lswitch1]) +OVN_NB_ADD_VTEP_PORT([br-test1], [br-vtep_lswitch1], [br-vtep], [lswitch1]) +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep br-vtep_lswitch1`"]) +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep_vtep_ls2`"]) + +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p1-l`"]) + +# Simulate peer on physical port to populate Ucast_Macs_Local table +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep_vtep_ls2 "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ee:02 actions=output:0100-p1-l"]) + +# Wait Ucast_Macs_Local is updated with local MAC +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep aa:bb:cc:dd:ee:02`"]) + +# Remove MAC from Ucast_Macs_Local +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep_vtep_ls1 "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ee:01"]) + +# Check that first MAC is removed from OVNSB, but second is still there +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding mac='"aa:bb:cc:dd:ee:01"'`"]) +OVS_WAIT_UNTIL([test -n "`ovn-sbctl find Mac_Binding mac='"aa:bb:cc:dd:ee:02"'`"]) + +# Remove second MAC from Ucast_Macs_Local +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep_vtep_ls2 "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ee:02"]) + +# Check that second MAC is removed from OVNSB +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding mac='"aa:bb:cc:dd:ee:01"'`"]) +OVS_WAIT_UNTIL([test -z "`ovn-sbctl find Mac_Binding mac='"aa:bb:cc:dd:ee:02"'`"]) + +# Start second VTEP chassis +OVN_CONTROLLER_VTEP_START(br-vtep1, 1.2.3.5) + +# creates the logical switch in vtep and adds the corresponding logical +# # port to 'br-test'. +AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep1 p0 100 lswitch0]) +OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep1_lswitch0], [br-vtep1], [lswitch0]) +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep br-vtep1_lswitch0`"]) +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep1_vtep_ls1`"]) + +# creates the logical switch in vtep and adds the corresponding logical +# # # port to 'br-test'. +AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep1 p1 100 lswitch1]) +OVN_NB_ADD_VTEP_PORT([br-test1], [br-vtep1_lswitch1], [br-vtep1], [lswitch1]) +OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep br-vtep1_lswitch1`"]) +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show | grep br-vtep1_vtep_ls2`"]) + +#sleep 3000 + +# Wait bingins are activated +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p0-l`"]) +OVS_WAIT_UNTIL([test -n "`ovs-vsctl show |grep 0100-p1-l`"]) + +# Simulate peer on physical port to populate Ucast_Macs_Local table on br-vtep1 +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep1_vtep_ls1 "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ff:01 actions=output:0100-p0-l"]) +AT_CHECK([ovs-ofctl -O OpenFlow14 add-flow br-vtep1_vtep_ls2 "cookie=0x5000,table=mac_learning_table,priority=1000,dl_dst=aa:bb:cc:dd:ff:02 actions=output:0100-p1-l"]) + +# Wait Ucast_Macs_Local is updated with local MAC +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep aa:bb:cc:dd:ff:01`"]) +OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Local | grep aa:bb:cc:dd:ff:02`"]) + +# Switch context +as + +# Make sure we see +OVS_WAIT_UNTIL([test -n "`vtep-ctl list-remote-macs lswitch0 | grep aa:bb:cc:dd:ff:01`"]) +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch1 | grep aa:bb:cc:dd:ff:01`"]) + +OVS_WAIT_UNTIL([test -n "`vtep-ctl list-remote-macs lswitch1 | grep aa:bb:cc:dd:ff:02`"]) +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch0 | grep aa:bb:cc:dd:ff:02`"]) + +# Switch context to br-vtep1 +as br-vtep1 + +# Remove local MACs +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep1_vtep_ls1 "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ff:01"]) +AT_CHECK([ovs-ofctl -O OpenFlow14 del-flows br-vtep1_vtep_ls2 "table=mac_learning_table,dl_dst=aa:bb:cc:dd:ff:02"]) + +# Switch context +as + +# Ensure remote MACs gone +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch0 | grep aa:bb:cc:dd:ff:01`"]) +OVS_WAIT_UNTIL([test -z "`vtep-ctl list-remote-macs lswitch1 | grep aa:bb:cc:dd:ff:02`"]) + +OVN_CONTROLLER_VTEP_STOP +AT_CLEANUP + # Tests vtep module 'Mcast_Macs_Remote's. AT_SETUP([ovn-controller-vtep - vtep-Mcast_Macs_Remote]) ovn_start
With this patch ovn-controller-vtep start synchronizing local MAC database to Mac_Binding OVN SB table. Which helps to resolve couple of problems: 1. Connectivity between hosts plugged to different switches. Currently OVN does not create tunnel between hardware switches even when they have ports plugged to same network. With this patch ovn-controller-vtep checking for Mac_Binding OVN SB database and setup required tunnels between switches. 2. OVN rely on broadcasts when send data to remote VTEP MACs. Since now we propagate switch local MAC table to SB database, later this information may be used by ovn-controller instances to create direct rules for hosts discovered from remote VTEPs. Signed-off-by: Vasyl Saienko <vsaienko@mirantis.com> --- controller-vtep/ovn-controller-vtep.c | 9 + controller-vtep/vtep.c | 301 +++++++++++++++++++++++++- tests/ovn-controller-vtep.at | 124 ++++++++++- 3 files changed, 425 insertions(+), 9 deletions(-)