| Message ID | 20240122172138.203956-1-naveen.yerramneni@nutanix.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev] Add "disable_arp_nd_rsp" option to LSP | expand |
On Mon, Jan 22, 2024 at 12:22 PM Naveen Yerramneni < naveen.yerramneni@nutanix.com> wrote: > This option can be used to enable/disable arp/nd reply flows. > > Usecase: > ========= > It is useful to reduce packet loss when VM is being migrated to > It may indeed be useful to be able to disable ARP responder for a LS/port. I am wondering if you have details about your packet loss issues when migrating a VM. Could you please confirm that we are talking about live migration (e.g. through libvirt) and that you already use multichassis port bindings to host the same port on multiple chassis (on source and destination)? In this case, OVN will set up flows that will clone (flood) traffic to both locations proactively, for the moment when your hypervisor switches running the VM from source to destination. You should not observe (significant) packet losses in this scenario. > different AZ via VXLAN tunnel. Port is configured in both AZs > on different logical switches which are sharing same IP subnet. > This snippet above suggests to me that you migrate between different logical switch ports? Could you please elaborate on how you set up your overlay connectivity for the VM? The reason I ask is because live migration reuses the same LSP, only changing the chassis that host(s) the LSP. > In reality, the port is active on only one logical switch. > Skipping ARP/ND responder and letting the ARP/ND get flooded to > learn the location of the port. > > Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> > --- > northd/northd.c | 10 +++++++++- > tests/ovn-northd.at | 31 +++++++++++++++++++++++++++++++ > 2 files changed, 40 insertions(+), 1 deletion(-) > > diff --git a/northd/northd.c b/northd/northd.c > index 952f8200d..4e070c0fe 100644 > --- a/northd/northd.c > +++ b/northd/northd.c > @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct > nbrec_logical_switch_port *nbsp) > return smap_get_bool( ->options, "localnet_learn_fdb", false); > } > > +static bool > +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) > +{ > + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); > +} > + > static bool > lsp_is_type_changed(const struct sbrec_port_binding *sb, > const struct nbrec_logical_switch_port *nbsp, > @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct > ovn_port *op, > return; > } > > - if (lsp_is_external(op->nbsp) || op->has_unknown) { > + if (lsp_is_external(op->nbsp) || op->has_unknown || > + (!strcmp(op->nbsp->type, "") && > + lsp_disable_arp_nd_rsp(op->nbsp))) { > return; > } > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > index 9a0d418e4..9a36ee810 100644 > --- a/tests/ovn-northd.at > +++ b/tests/ovn-northd.at > @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl > | sed 's/table=./table=?/'], [0 > ]) > > > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) > +ovn_start NORTHD_TYPE > +ovn-nbctl ls-add S1 > +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 > +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 > 192.168.0.10 fd00::10" > + > +ovn-sbctl dump-flows S1 > S1flows > +AT_CAPTURE_FILE([S1flows]) > + > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], > [0], [dnl > + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == > 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) > + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst > == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == > "S1-vm1"), action=(next;) > + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == > 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = > 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = > 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = > inport; flags.loopback = 1; output;) > + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst > == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { > eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; > nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; > };) > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > +]) > + > +#Set the disable_arp_nd_rsp option and verify the flow > +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 > options:disable_arp_nd_rsp=true > + > +ovn-sbctl dump-flows S1 > S1flows > +AT_CAPTURE_FILE([S1flows]) > + > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], > [0], [dnl > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > +]) > + > AT_CLEANUP > ]) > -- > 2.36.6 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > >
> On 29-Jan-2024, at 9:11 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > On Mon, Jan 22, 2024 at 12:22 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > This option can be used to enable/disable arp/nd reply flows. > > Usecase: > ========= > It is useful to reduce packet loss when VM is being migrated to > > It may indeed be useful to be able to disable ARP responder for a LS/port. > > I am wondering if you have details about your packet loss issues when migrating a VM. Could you please confirm that we are talking about live migration (e.g. through libvirt) and that you already use multichassis port bindings to host the same port on multiple chassis (on source and destination)? In this case, OVN will set up flows that will clone (flood) traffic to both locations proactively, for the moment when your hypervisor switches running the VM from source to destination. You should not observe (significant) packet losses in this scenario. VM migration is happening between two different logical switches (i.e., ports are different) hence requested-chassis option is not helpful here. In this case, same VLAN is stretched using VXLAN (external VTEP devices). Packet loss is observed in 2 cases: 1. When port is configured on the destination but migration is still in progress. Patch raised for this - https://www.mail-archive.com/ovs-dev@openvswitch.org/msg82745.html 2. When VM sends GARP packet post migration and port is not yet deleted on the source side then, source side logical switch responds to GARP. This makes the intermediate VTEP devices to incorrectly learn the location of the port. Skipping ARP/ND responder and letting the ARP/ND get flooded to learn the location of the port properly. > different AZ via VXLAN tunnel. Port is configured in both AZs > on different logical switches which are sharing same IP subnet. > > This snippet above suggests to me that you migrate between different logical switch ports? Could you please elaborate on how you set up your overlay connectivity for the VM? > > The reason I ask is because live migration reuses the same LSP, only changing the chassis that host(s) the LSP. VM migration is happening between two different logical switches (i.e., ports are different). In this case, same VLAN is stretched using VXLAN (external VTEP devices). > > In reality, the port is active on only one logical switch. > Skipping ARP/ND responder and letting the ARP/ND get flooded to > learn the location of the port. > > Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> > --- > northd/northd.c | 10 +++++++++- > tests/ovn-northd.at [ovn-northd.at] | 31 +++++++++++++++++++++++++++++++ > 2 files changed, 40 insertions(+), 1 deletion(-) > > diff --git a/northd/northd.c b/northd/northd.c > index 952f8200d..4e070c0fe 100644 > --- a/northd/northd.c > +++ b/northd/northd.c > @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct nbrec_logical_switch_port *nbsp) > return smap_get_bool( ->options, "localnet_learn_fdb", false); > } > > +static bool > +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) > +{ > + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); > +} > + > static bool > lsp_is_type_changed(const struct sbrec_port_binding *sb, > const struct nbrec_logical_switch_port *nbsp, > @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, > return; > } > > - if (lsp_is_external(op->nbsp) || op->has_unknown) { > + if (lsp_is_external(op->nbsp) || op->has_unknown || > + (!strcmp(op->nbsp->type, "") && > + lsp_disable_arp_nd_rsp(op->nbsp))) { > return; > } > > diff --git a/tests/ovn-northd.at [ovn-northd.at] b/tests/ovn-northd.at [ovn-northd.at] > index 9a0d418e4..9a36ee810 100644 > --- a/tests/ovn-northd.at [ovn-northd.at] > +++ b/tests/ovn-northd.at [ovn-northd.at] > @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl | sed 's/table=./table=?/'], [0 > ]) > > > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) > +ovn_start NORTHD_TYPE > +ovn-nbctl ls-add S1 > +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 > +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 192.168.0.10 fd00::10" > + > +ovn-sbctl dump-flows S1 > S1flows > +AT_CAPTURE_FILE([S1flows]) > + > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) > + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == "S1-vm1"), action=(next;) > + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = inport; flags.loopback = 1; output;) > + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; };) > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > +]) > + > +#Set the disable_arp_nd_rsp option and verify the flow > +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 options:disable_arp_nd_rsp=true > + > +ovn-sbctl dump-flows S1 > S1flows > +AT_CAPTURE_FILE([S1flows]) > + > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > +]) > + > AT_CLEANUP > ]) > -- > 2.36.6 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev [mail.openvswitch.org]
On Tue, Jan 30, 2024 at 10:52 PM Naveen Yerramneni < naveen.yerramneni@nutanix.com> wrote: > > > > On 29-Jan-2024, at 9:11 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > > > On Mon, Jan 22, 2024 at 12:22 PM Naveen Yerramneni < > naveen.yerramneni@nutanix.com> wrote: > > This option can be used to enable/disable arp/nd reply flows. > > > > Usecase: > > ========= > > It is useful to reduce packet loss when VM is being migrated to > > > > It may indeed be useful to be able to disable ARP responder for a > LS/port. > > > > I am wondering if you have details about your packet loss issues when > migrating a VM. Could you please confirm that we are talking about live > migration (e.g. through libvirt) and that you already use multichassis port > bindings to host the same port on multiple chassis (on source and > destination)? In this case, OVN will set up flows that will clone (flood) > traffic to both locations proactively, for the moment when your hypervisor > switches running the VM from source to destination. You should not observe > (significant) packet losses in this scenario. > > VM migration is happening between two different logical switches (i.e., > ports are different) hence requested-chassis option is not helpful here. In > this case, same VLAN is stretched using VXLAN (external VTEP devices). > Thanks for getting back to me. It's interesting to see how other people deal with the problem. I now see that you are dealing with VLAN stretched over VTEPs that are not under control of OVN and hence cannot be educated about port location. If I may ask, in your scenario, how do you retain identity for a VM interface (MAC, IPs) while swapping LSPs that back the interface? Do you set addresses for both LSPs to the same MAC/IP tuple? > > Packet loss is observed in 2 cases: > 1. When port is configured on the destination but migration is still in > progress. Patch raised for this - > https://www.mail-archive.com/ovs-dev@openvswitch.org/msg82745.html > 2. When VM sends GARP packet post migration and port is not yet deleted > on the source side then, source side logical switch responds to GARP. This > makes the intermediate VTEP devices to incorrectly learn the location of > the port. Skipping ARP/ND responder and letting the ARP/ND get flooded to > learn the location of the port properly. > > > > different AZ via VXLAN tunnel. Port is configured in both AZs > > on different logical switches which are sharing same IP subnet. > > > > This snippet above suggests to me that you migrate between different > logical switch ports? Could you please elaborate on how you set up your > overlay connectivity for the VM? > > > > The reason I ask is because live migration reuses the same LSP, only > changing the chassis that host(s) the LSP. > > VM migration is happening between two different logical switches (i.e., > ports are different). In this case, same VLAN is stretched using VXLAN > (external VTEP devices). > > > > > In reality, the port is active on only one logical switch. > > Skipping ARP/ND responder and letting the ARP/ND get flooded to > > learn the location of the port. > > > > Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> > > --- > > northd/northd.c | 10 +++++++++- > > tests/ovn-northd.at [ovn-northd.at] | 31 > +++++++++++++++++++++++++++++++ > > 2 files changed, 40 insertions(+), 1 deletion(-) > > > > diff --git a/northd/northd.c b/northd/northd.c > > index 952f8200d..4e070c0fe 100644 > > --- a/northd/northd.c > > +++ b/northd/northd.c > > @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct > nbrec_logical_switch_port *nbsp) > > return smap_get_bool( ->options, "localnet_learn_fdb", false); > > } > > > > +static bool > > +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) > > +{ > > + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); > > +} > > + > > static bool > > lsp_is_type_changed(const struct sbrec_port_binding *sb, > > const struct nbrec_logical_switch_port *nbsp, > > @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct > ovn_port *op, > > return; > > } > > > > - if (lsp_is_external(op->nbsp) || op->has_unknown) { > > + if (lsp_is_external(op->nbsp) || op->has_unknown || > > + (!strcmp(op->nbsp->type, "") && > > + lsp_disable_arp_nd_rsp(op->nbsp))) { > > return; > > } > > > > diff --git a/tests/ovn-northd.at [ovn-northd.at] b/tests/ovn-northd.at [ > ovn-northd.at] > > index 9a0d418e4..9a36ee810 100644 > > --- a/tests/ovn-northd.at [ovn-northd.at] > > +++ b/tests/ovn-northd.at [ovn-northd.at] > > @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep > pre_acl | sed 's/table=./table=?/'], [0 > > ]) > > > > > > +AT_CLEANUP > > +]) > > + > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) > > +ovn_start NORTHD_TYPE > > +ovn-nbctl ls-add S1 > > +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 > > +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 > 192.168.0.10 fd00::10" > > + > > +ovn-sbctl dump-flows S1 > S1flows > > +AT_CAPTURE_FILE([S1flows]) > > + > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed > 's/table=../table=??/'], [0], [dnl > > + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == > 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) > > + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && > ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport > == "S1-vm1"), action=(next;) > > + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == > 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = > 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = > 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = > inport; flags.loopback = 1; output;) > > + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && > ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), > action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target > = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = > 1; output; };) > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), > action=(next;) > > +]) > > + > > +#Set the disable_arp_nd_rsp option and verify the flow > > +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 > options:disable_arp_nd_rsp=true > > + > > +ovn-sbctl dump-flows S1 > S1flows > > +AT_CAPTURE_FILE([S1flows]) > > + > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed > 's/table=../table=??/'], [0], [dnl > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), > action=(next;) > > +]) > > + > > AT_CLEANUP > > ]) > > -- > > 2.36.6 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev [ > mail.openvswitch.org] > >
> On 12-Feb-2024, at 8:36 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > On Tue, Jan 30, 2024 at 10:52 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > > > > On 29-Jan-2024, at 9:11 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > > > On Mon, Jan 22, 2024 at 12:22 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > > This option can be used to enable/disable arp/nd reply flows. > > > > Usecase: > > ========= > > It is useful to reduce packet loss when VM is being migrated to > > > > It may indeed be useful to be able to disable ARP responder for a LS/port. > > > > I am wondering if you have details about your packet loss issues when migrating a VM. Could you please confirm that we are talking about live migration (e.g. through libvirt) and that you already use multichassis port bindings to host the same port on multiple chassis (on source and destination)? In this case, OVN will set up flows that will clone (flood) traffic to both locations proactively, for the moment when your hypervisor switches running the VM from source to destination. You should not observe (significant) packet losses in this scenario. > > VM migration is happening between two different logical switches (i.e., ports are different) hence requested-chassis option is not helpful here. In this case, same VLAN is stretched using VXLAN (external VTEP devices). > > Thanks for getting back to me. It's interesting to see how other people deal with the problem. I now see that you are dealing with VLAN stretched over VTEPs that are not under control of OVN and hence cannot be educated about port location. > > If I may ask, in your scenario, how do you retain identity for a VM interface (MAC, IPs) while swapping LSPs that back the interface? Do you set addresses for both LSPs to the same MAC/IP tuple? Yes, MAC, IP are same for both LSPs. > > Packet loss is observed in 2 cases: > 1. When port is configured on the destination but migration is still in progress. Patch raised for this - https://www.mail-archive.com/ovs-dev@openvswitch.org/msg82745.html [mail-archive.com] > 2. When VM sends GARP packet post migration and port is not yet deleted on the source side then, source side logical switch responds to GARP. This makes the intermediate VTEP devices to incorrectly learn the location of the port. Skipping ARP/ND responder and letting the ARP/ND get flooded to learn the location of the port properly. > > > > different AZ via VXLAN tunnel. Port is configured in both AZs > > on different logical switches which are sharing same IP subnet. > > > > This snippet above suggests to me that you migrate between different logical switch ports? Could you please elaborate on how you set up your overlay connectivity for the VM? > > > > The reason I ask is because live migration reuses the same LSP, only changing the chassis that host(s) the LSP. > > VM migration is happening between two different logical switches (i.e., ports are different). In this case, same VLAN is stretched using VXLAN (external VTEP devices). > > > > > In reality, the port is active on only one logical switch. > > Skipping ARP/ND responder and letting the ARP/ND get flooded to > > learn the location of the port. > > > > Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> > > --- > > northd/northd.c | 10 +++++++++- > > tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] | 31 +++++++++++++++++++++++++++++++ > > 2 files changed, 40 insertions(+), 1 deletion(-) > > > > diff --git a/northd/northd.c b/northd/northd.c > > index 952f8200d..4e070c0fe 100644 > > --- a/northd/northd.c > > +++ b/northd/northd.c > > @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct nbrec_logical_switch_port *nbsp) > > return smap_get_bool( ->options, "localnet_learn_fdb", false); > > } > > > > +static bool > > +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) > > +{ > > + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); > > +} > > + > > static bool > > lsp_is_type_changed(const struct sbrec_port_binding *sb, > > const struct nbrec_logical_switch_port *nbsp, > > @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, > > return; > > } > > > > - if (lsp_is_external(op->nbsp) || op->has_unknown) { > > + if (lsp_is_external(op->nbsp) || op->has_unknown || > > + (!strcmp(op->nbsp->type, "") && > > + lsp_disable_arp_nd_rsp(op->nbsp))) { > > return; > > } > > > > diff --git a/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] b/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > index 9a0d418e4..9a36ee810 100644 > > --- a/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > +++ b/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl | sed 's/table=./table=?/'], [0 > > ]) > > > > > > +AT_CLEANUP > > +]) > > + > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) > > +ovn_start NORTHD_TYPE > > +ovn-nbctl ls-add S1 > > +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 > > +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 192.168.0.10 fd00::10" > > + > > +ovn-sbctl dump-flows S1 > S1flows > > +AT_CAPTURE_FILE([S1flows]) > > + > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > > + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) > > + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == "S1-vm1"), action=(next;) > > + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = inport; flags.loopback = 1; output;) > > + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; };) > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > > +]) > > + > > +#Set the disable_arp_nd_rsp option and verify the flow > > +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 options:disable_arp_nd_rsp=true > > + > > +ovn-sbctl dump-flows S1 > S1flows > > +AT_CAPTURE_FILE([S1flows]) > > + > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > > +]) > > + > > AT_CLEANUP > > ]) > > -- > > 2.36.6 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev [mail.openvswitch.org] [mail.openvswitch.org [mail.openvswitch.org]]
On Thu, Feb 15, 2024 at 8:46 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > > > > > On 12-Feb-2024, at 8:36 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > > > On Tue, Jan 30, 2024 at 10:52 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > > > > > > > On 29-Jan-2024, at 9:11 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote: > > > > > > On Mon, Jan 22, 2024 at 12:22 PM Naveen Yerramneni <naveen.yerramneni@nutanix.com> wrote: > > > This option can be used to enable/disable arp/nd reply flows. > > > > > > Usecase: > > > ========= > > > It is useful to reduce packet loss when VM is being migrated to > > > > > > It may indeed be useful to be able to disable ARP responder for a LS/port. > > > > > > I am wondering if you have details about your packet loss issues when migrating a VM. Could you please confirm that we are talking about live migration (e.g. through libvirt) and that you already use multichassis port bindings to host the same port on multiple chassis (on source and destination)? In this case, OVN will set up flows that will clone (flood) traffic to both locations proactively, for the moment when your hypervisor switches running the VM from source to destination. You should not observe (significant) packet losses in this scenario. > > > > VM migration is happening between two different logical switches (i.e., ports are different) hence requested-chassis option is not helpful here. In this case, same VLAN is stretched using VXLAN (external VTEP devices). > > > > Thanks for getting back to me. It's interesting to see how other people deal with the problem. I now see that you are dealing with VLAN stretched over VTEPs that are not under control of OVN and hence cannot be educated about port location. > > > > If I may ask, in your scenario, how do you retain identity for a VM interface (MAC, IPs) while swapping LSPs that back the interface? Do you set addresses for both LSPs to the same MAC/IP tuple? > > Yes, MAC, IP are same for both LSPs. > > > > > Packet loss is observed in 2 cases: > > 1. When port is configured on the destination but migration is still in progress. Patch raised for this - https://www.mail-archive.com/ovs-dev@openvswitch.org/msg82745.html [mail-archive.com] > > 2. When VM sends GARP packet post migration and port is not yet deleted on the source side then, source side logical switch responds to GARP. This makes the intermediate VTEP devices to incorrectly learn the location of the port. Skipping ARP/ND responder and letting the ARP/ND get flooded to learn the location of the port properly. > > > > > > > different AZ via VXLAN tunnel. Port is configured in both AZs > > > on different logical switches which are sharing same IP subnet. > > > > > > This snippet above suggests to me that you migrate between different logical switch ports? Could you please elaborate on how you set up your overlay connectivity for the VM? > > > > > > The reason I ask is because live migration reuses the same LSP, only changing the chassis that host(s) the LSP. > > > > VM migration is happening between two different logical switches (i.e., ports are different). In this case, same VLAN is stretched using VXLAN (external VTEP devices). > > > > > > > > In reality, the port is active on only one logical switch. > > > Skipping ARP/ND responder and letting the ARP/ND get flooded to > > > learn the location of the port. > > > > > > Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> > > > --- > > > northd/northd.c | 10 +++++++++- > > > tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] | 31 +++++++++++++++++++++++++++++++ > > > 2 files changed, 40 insertions(+), 1 deletion(-) > > > > > > diff --git a/northd/northd.c b/northd/northd.c > > > index 952f8200d..4e070c0fe 100644 > > > --- a/northd/northd.c > > > +++ b/northd/northd.c > > > @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct nbrec_logical_switch_port *nbsp) > > > return smap_get_bool( ->options, "localnet_learn_fdb", false); > > > } > > > > > > +static bool > > > +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) > > > +{ > > > + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); > > > +} > > > + > > > static bool > > > lsp_is_type_changed(const struct sbrec_port_binding *sb, > > > const struct nbrec_logical_switch_port *nbsp, > > > @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, > > > return; > > > } > > > > > > - if (lsp_is_external(op->nbsp) || op->has_unknown) { > > > + if (lsp_is_external(op->nbsp) || op->has_unknown || > > > + (!strcmp(op->nbsp->type, "") && > > > + lsp_disable_arp_nd_rsp(op->nbsp))) { > > > return; > > > } > > > > > > diff --git a/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] b/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > > index 9a0d418e4..9a36ee810 100644 > > > --- a/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > > +++ b/tests/ovn-northd.at [ovn-northd.at] [ovn-northd.at [ovn-northd.at]] > > > @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl | sed 's/table=./table=?/'], [0 > > > ]) > > > > > > > > > +AT_CLEANUP > > > +]) > > > + > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) > > > +ovn_start NORTHD_TYPE > > > +ovn-nbctl ls-add S1 > > > +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 > > > +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 192.168.0.10 fd00::10" > > > + > > > +ovn-sbctl dump-flows S1 > S1flows > > > +AT_CAPTURE_FILE([S1flows]) > > > + > > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > > > + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) > > > + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == "S1-vm1"), action=(next;) > > > + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = inport; flags.loopback = 1; output;) > > > + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; };) > > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > > > +]) > > > + > > > +#Set the disable_arp_nd_rsp option and verify the flow > > > +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 options:disable_arp_nd_rsp=true > > > + > > > +ovn-sbctl dump-flows S1 > S1flows > > > +AT_CAPTURE_FILE([S1flows]) > > > + > > > +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl > > > + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) > > > +]) > > > + > > > AT_CLEANUP > > > ]) Thanks for the patch. I applied this patch to the main with a few changes. -------------------------------------- diff --git a/NEWS b/NEWS index 680ae49cbc..c0131ceee7 100644 --- a/NEWS +++ b/NEWS @@ -3,6 +3,8 @@ Post v24.03.0 - Added a new logical switch port option "pkt_clone_type". If the value is set to "mc_unknown", packets destined to the port gets cloned to all unknown ports connected to the same Logical Switch. + - Added a new logical switch port option "disable_arp_nd_rsp" to + disable adding the ARP responder flows if set to true. OVN v24.03.0 - xx xxx xxxx -------------------------- diff --git a/northd/northd.c b/northd/northd.c index ab4c1c30f6..2c3560ce2d 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8821,7 +8821,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, } else { /* * Add ARP/ND reply flows if either the - * - port is up and it doesn't have 'unknown' address defined or + * - port is up and it doesn't have 'unknown' address defined or it + * doesn't have the option disable_arp_nd_rsp=true. * - port type is router or * - port type is localport */ @@ -8832,8 +8833,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, } if (lsp_is_external(op->nbsp) || op->has_unknown || - (!strcmp(op->nbsp->type, "") && - lsp_disable_arp_nd_rsp(op->nbsp))) { + (!op->nbsp->type[0] && lsp_disable_arp_nd_rsp(op->nbsp))) { return; } diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 98799e91c1..17b4141441 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -1458,8 +1458,9 @@ output; ignore_lsp_down</code> is configured as true in <code>options</code> column of <code>NB_Global</code> table of the <code>Northbound</code> database), for logical ports of type <code>virtual</code>, for - logical ports with 'unknown' address set and for logical ports of - a logical switch configured with + logical ports with 'unknown' address set, for logical ports with + the <code>options:disable_arp_nd_rsp=true</code> and for logical + ports of a logical switch configured with <code>other_config:vlan-passthru=true</code>. </p> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index bdefd28a05..c189dcccc1 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9233,12 +9233,12 @@ ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 192.168.0.10 fd ovn-sbctl dump-flows S1 > S1flows AT_CAPTURE_FILE([S1flows]) -AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | ovn_strip_lflows], [0], [dnl + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == "S1-vm1"), action=(next;) table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = inport; flags.loopback = 1; output;) table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; };) - table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) ]) #Set the disable_arp_nd_rsp option and verify the flow @@ -9247,7 +9247,7 @@ ovn-nbctl --wait=sb set logical_switch_port S1-vm1 options:disable_arp_nd_rsp=tr ovn-sbctl dump-flows S1 > S1flows AT_CAPTURE_FILE([S1flows]) -AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | ovn_strip_lflows], [0], [dnl table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) ]) -------------------------------------- Thanks Numan > > > -- > > > 2.36.6 > > > > > > _______________________________________________ > > > dev mailing list > > > dev@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev [mail.openvswitch.org] [mail.openvswitch.org [mail.openvswitch.org]] > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff --git a/northd/northd.c b/northd/northd.c index 952f8200d..4e070c0fe 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -1844,6 +1844,12 @@ localnet_can_learn_mac(const struct nbrec_logical_switch_port *nbsp) return smap_get_bool( ->options, "localnet_learn_fdb", false); } +static bool +lsp_disable_arp_nd_rsp(const struct nbrec_logical_switch_port *nbsp) +{ + return smap_get_bool( ->options, "disable_arp_nd_rsp", false); +} + static bool lsp_is_type_changed(const struct sbrec_port_binding *sb, const struct nbrec_logical_switch_port *nbsp, @@ -9921,7 +9927,9 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, return; } - if (lsp_is_external(op->nbsp) || op->has_unknown) { + if (lsp_is_external(op->nbsp) || op->has_unknown || + (!strcmp(op->nbsp->type, "") && + lsp_disable_arp_nd_rsp(op->nbsp))) { return; } diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9a0d418e4..9a36ee810 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11094,5 +11094,36 @@ AT_CHECK([ovn-sbctl dump-flows S1 | grep pre_acl | sed 's/table=./table=?/'], [0 ]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([check options:disable_arp_nd_rsp for LSP]) +ovn_start NORTHD_TYPE +ovn-nbctl ls-add S1 +ovn-nbctl --wait=sb lsp-add S1 S1-vm1 +ovn-nbctl --wait=sb lsp-set-addresses S1-vm1 "50:54:00:00:00:010 192.168.0.10 fd00::10" + +ovn-sbctl dump-flows S1 > S1flows +AT_CAPTURE_FILE([S1flows]) + +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl + table=??(ls_in_arp_rsp ), priority=100 , match=(arp.tpa == 192.168.0.10 && arp.op == 1 && inport == "S1-vm1"), action=(next;) + table=??(ls_in_arp_rsp ), priority=100 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10 && inport == "S1-vm1"), action=(next;) + table=??(ls_in_arp_rsp ), priority=50 , match=(arp.tpa == 192.168.0.10 && arp.op == 1), action=(eth.dst = eth.src; eth.src = 50:54:00:00:00:10; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 50:54:00:00:00:10; arp.tpa = arp.spa; arp.spa = 192.168.0.10; outport = inport; flags.loopback = 1; output;) + table=??(ls_in_arp_rsp ), priority=50 , match=(nd_ns && ip6.dst == {fd00::10, ff02::1:ff00:10} && nd.target == fd00::10), action=(nd_na { eth.src = 50:54:00:00:00:10; ip6.src = fd00::10; nd.target = fd00::10; nd.tll = 50:54:00:00:00:10; outport = inport; flags.loopback = 1; output; };) + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) +]) + +#Set the disable_arp_nd_rsp option and verify the flow +ovn-nbctl --wait=sb set logical_switch_port S1-vm1 options:disable_arp_nd_rsp=true + +ovn-sbctl dump-flows S1 > S1flows +AT_CAPTURE_FILE([S1flows]) + +AT_CHECK([grep -e "ls_in_arp_rsp" S1flows | sed 's/table=../table=??/'], [0], [dnl + table=??(ls_in_arp_rsp ), priority=0 , match=(1), action=(next;) +]) + AT_CLEANUP ])
This option can be used to enable/disable arp/nd reply flows. Usecase: ========= It is useful to reduce packet loss when VM is being migrated to different AZ via VXLAN tunnel. Port is configured in both AZs on different logical switches which are sharing same IP subnet. In reality, the port is active on only one logical switch. Skipping ARP/ND responder and letting the ARP/ND get flooded to learn the location of the port. Signed-off-by: Naveen Yerramneni <naveen.yerramneni@nutanix.com> --- northd/northd.c | 10 +++++++++- tests/ovn-northd.at | 31 +++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-)