| Message ID | 20220622162113.1509569-1-xsimonar@redhat.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev] tests: fixed flaky test localnet connectivity with multiple requested-chassis | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | fail | github build: failed |
| ovsrobot/github-robot-_ovn-kubernetes | success | github build: passed |
On 6/22/22 18:21, Xavier Simonart wrote: > The test was failing randomly (on highly loaded systems) mainly because > the MAC address of the migrator port in the main switch was migrating at > unexpected times. > In addition, a few undefined variables were fixed, and sleep was removed / > replaced by ovs_wait actions. > > Signed-off-by: Xavier Simonart <xsimonar@redhat.com> > --- > tests/ovn.at | 22 ++++++++++++++++------ > 1 file changed, 16 insertions(+), 6 deletions(-) > > diff --git a/tests/ovn.at b/tests/ovn.at > index bfaa41962..12d1f8667 100644 > --- a/tests/ovn.at > +++ b/tests/ovn.at > @@ -14691,10 +14691,10 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator > wait_for_ports_up > > # advertise location of ports through localnet port > -send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa > -send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_tpa > -send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_tpa > -send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_tpa > +send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa > +send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_spa > +send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_spa > +send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_spa > reset_env > > # check that... > @@ -14840,6 +14840,12 @@ echo $request >> hv3/third.expected > > check_packets > > +# Wait for MAC address of migrator to be on hv1 related port in main switch. > +# Hence the MAC will not migrate back unexpectedly later. > +p1=$(as main ovs-ofctl show n1 | grep hv1_br-phys | awk '{print int($1)}') > +p2=$(as main ovs-ofctl show n1 | grep hv2_br-phys | awk '{print int($1)}') > +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p1]) Nit: extra whitespace before last pipe. > + > # Complete migration: destination is bound > check ovn-nbctl lsp-set-options migrator requested-chassis=hv2 > wait_column "$hv2_uuid" Port_Binding chassis logical_port=migrator > @@ -14849,12 +14855,16 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator > wait_for_ports_up > > check ovn-nbctl --wait=hv sync > -sleep 1 > +OVS_WAIT_UNTIL([test `as hv2 ovs-vsctl get Interface migrator external_ids:ovn-installed` = '"true"']) > > # advertise new location of the port through localnet port > -send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa > +send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa > + > reset_env > > +# Wait for MAC address of migrator to be on hv2 port in main switch > +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p2]) Nit: extra whitespace before last pipe. I'm guessing both can be fixed at apply time, therefore: Acked-by: Dumitru Ceara <dceara@redhat.com> Thanks!
Thank you. Acked-by: Ihar Hrachyshka <ihrachys@redhat.com> On Wed, Jun 22, 2022 at 12:21 PM Xavier Simonart <xsimonar@redhat.com> wrote: > > The test was failing randomly (on highly loaded systems) mainly because > the MAC address of the migrator port in the main switch was migrating at > unexpected times. > In addition, a few undefined variables were fixed, and sleep was removed / > replaced by ovs_wait actions. > > Signed-off-by: Xavier Simonart <xsimonar@redhat.com> > --- > tests/ovn.at | 22 ++++++++++++++++------ > 1 file changed, 16 insertions(+), 6 deletions(-) > > diff --git a/tests/ovn.at b/tests/ovn.at > index bfaa41962..12d1f8667 100644 > --- a/tests/ovn.at > +++ b/tests/ovn.at > @@ -14691,10 +14691,10 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator > wait_for_ports_up > > # advertise location of ports through localnet port > -send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa > -send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_tpa > -send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_tpa > -send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_tpa > +send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa > +send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_spa > +send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_spa > +send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_spa > reset_env > > # check that... > @@ -14840,6 +14840,12 @@ echo $request >> hv3/third.expected > > check_packets > > +# Wait for MAC address of migrator to be on hv1 related port in main switch. > +# Hence the MAC will not migrate back unexpectedly later. > +p1=$(as main ovs-ofctl show n1 | grep hv1_br-phys | awk '{print int($1)}') > +p2=$(as main ovs-ofctl show n1 | grep hv2_br-phys | awk '{print int($1)}') > +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p1]) > + > # Complete migration: destination is bound > check ovn-nbctl lsp-set-options migrator requested-chassis=hv2 > wait_column "$hv2_uuid" Port_Binding chassis logical_port=migrator > @@ -14849,12 +14855,16 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator > wait_for_ports_up > > check ovn-nbctl --wait=hv sync > -sleep 1 > +OVS_WAIT_UNTIL([test `as hv2 ovs-vsctl get Interface migrator external_ids:ovn-installed` = '"true"']) > > # advertise new location of the port through localnet port > -send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa > +send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa > + > reset_env > > +# Wait for MAC address of migrator to be on hv2 port in main switch > +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p2]) > + > # check that... > # unicast from Third doesn't arrive to hv1:Migrator > # unicast from Third arrives to hv2:Migrator > -- > 2.31.1 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
Thanks everyone, I pushed this to main and branch-22.06 On 6/29/22 18:52, Ihar Hrachyshka wrote: > Thank you. > > Acked-by: Ihar Hrachyshka <ihrachys@redhat.com> > > On Wed, Jun 22, 2022 at 12:21 PM Xavier Simonart <xsimonar@redhat.com> wrote: >> >> The test was failing randomly (on highly loaded systems) mainly because >> the MAC address of the migrator port in the main switch was migrating at >> unexpected times. >> In addition, a few undefined variables were fixed, and sleep was removed / >> replaced by ovs_wait actions. >> >> Signed-off-by: Xavier Simonart <xsimonar@redhat.com> >> --- >> tests/ovn.at | 22 ++++++++++++++++------ >> 1 file changed, 16 insertions(+), 6 deletions(-) >> >> diff --git a/tests/ovn.at b/tests/ovn.at >> index bfaa41962..12d1f8667 100644 >> --- a/tests/ovn.at >> +++ b/tests/ovn.at >> @@ -14691,10 +14691,10 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator >> wait_for_ports_up >> >> # advertise location of ports through localnet port >> -send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa >> -send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_tpa >> -send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_tpa >> -send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_tpa >> +send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa >> +send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_spa >> +send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_spa >> +send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_spa >> reset_env >> >> # check that... >> @@ -14840,6 +14840,12 @@ echo $request >> hv3/third.expected >> >> check_packets >> >> +# Wait for MAC address of migrator to be on hv1 related port in main switch. >> +# Hence the MAC will not migrate back unexpectedly later. >> +p1=$(as main ovs-ofctl show n1 | grep hv1_br-phys | awk '{print int($1)}') >> +p2=$(as main ovs-ofctl show n1 | grep hv2_br-phys | awk '{print int($1)}') >> +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p1]) >> + >> # Complete migration: destination is bound >> check ovn-nbctl lsp-set-options migrator requested-chassis=hv2 >> wait_column "$hv2_uuid" Port_Binding chassis logical_port=migrator >> @@ -14849,12 +14855,16 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator >> wait_for_ports_up >> >> check ovn-nbctl --wait=hv sync >> -sleep 1 >> +OVS_WAIT_UNTIL([test `as hv2 ovs-vsctl get Interface migrator external_ids:ovn-installed` = '"true"']) >> >> # advertise new location of the port through localnet port >> -send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa >> +send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa >> + >> reset_env >> >> +# Wait for MAC address of migrator to be on hv2 port in main switch >> +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p2]) >> + >> # check that... >> # unicast from Third doesn't arrive to hv1:Migrator >> # unicast from Third arrives to hv2:Migrator >> -- >> 2.31.1 >> >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
diff --git a/tests/ovn.at b/tests/ovn.at index bfaa41962..12d1f8667 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -14691,10 +14691,10 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator wait_for_ports_up # advertise location of ports through localnet port -send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa -send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_tpa -send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_tpa -send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_tpa +send_garp hv1 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa +send_garp hv1 first 000000000001 ffffffffffff $first_spa $first_spa +send_garp hv2 second 000000000002 ffffffffffff $second_spa $second_spa +send_garp hv3 third 000000000003 ffffffffffff $third_spa $third_spa reset_env # check that... @@ -14840,6 +14840,12 @@ echo $request >> hv3/third.expected check_packets +# Wait for MAC address of migrator to be on hv1 related port in main switch. +# Hence the MAC will not migrate back unexpectedly later. +p1=$(as main ovs-ofctl show n1 | grep hv1_br-phys | awk '{print int($1)}') +p2=$(as main ovs-ofctl show n1 | grep hv2_br-phys | awk '{print int($1)}') +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p1]) + # Complete migration: destination is bound check ovn-nbctl lsp-set-options migrator requested-chassis=hv2 wait_column "$hv2_uuid" Port_Binding chassis logical_port=migrator @@ -14849,12 +14855,16 @@ wait_column "" Port_Binding requested_additional_chassis logical_port=migrator wait_for_ports_up check ovn-nbctl --wait=hv sync -sleep 1 +OVS_WAIT_UNTIL([test `as hv2 ovs-vsctl get Interface migrator external_ids:ovn-installed` = '"true"']) # advertise new location of the port through localnet port -send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_spa $migrator_tpa +send_garp hv2 migrator 0000000000ff ffffffffffff $migrator_tpa $migrator_tpa + reset_env +# Wait for MAC address of migrator to be on hv2 port in main switch +OVS_WAIT_UNTIL([test x`as main ovs-appctl fdb/show n1 | grep 00:00:00:00:00:ff | awk '{print $1}'` = x$p2]) + # check that... # unicast from Third doesn't arrive to hv1:Migrator # unicast from Third arrives to hv2:Migrator
The test was failing randomly (on highly loaded systems) mainly because the MAC address of the migrator port in the main switch was migrating at unexpected times. In addition, a few undefined variables were fixed, and sleep was removed / replaced by ovs_wait actions. Signed-off-by: Xavier Simonart <xsimonar@redhat.com> --- tests/ovn.at | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-)