[ovs-dev,v1] tests: Include ovn-macros for system testsuites.
diff mbox series

Message ID 1558817387-6277-1-git-send-email-dlu998@gmail.com
State New
Headers show
Series
  • [ovs-dev,v1] tests: Include ovn-macros for system testsuites.
Related show

Commit Message

Darrell Ball May 25, 2019, 8:49 p.m. UTC
ovn-macros are needed to run the OVN system tests.

Signed-off-by: Darrell Ball <dlu998@gmail.com>
---
 tests/system-kmod-testsuite.at      | 1 +
 tests/system-userspace-testsuite.at | 1 +
 2 files changed, 2 insertions(+)

Comments

0-day Robot May 25, 2019, 9:17 p.m. UTC | #1
Bleep bloop.  Greetings Darrell Ball, I am a robot and I have tried out your patch.
Thanks for your contribution.

I encountered some error that I wasn't expecting.  See the details below.


build:
  sysconfdir='/usr/local/etc' \
  bindir='/usr/local/bin' \
  sbindir='/usr/local/sbin'
mv northd/ovn-northd.8.tmp northd/ovn-northd.8
printf '\043 Generated automatically -- do not modify!    -*- buffer-read-only: t -*-\n' > version.py.tmp && \
echo 'VERSION = "2.11.90"' >> version.py.tmp && \
if cmp -s version.py.tmp python/ovs/version.py; then touch python/ovs/version.py; rm version.py.tmp; else mv version.py.tmp python/ovs/version.py; fi
:;{ \
  echo '# Signature of the current package.' && \
  echo 'm4_define([AT_PACKAGE_NAME],      [openvswitch])' && \
  echo 'm4_define([AT_PACKAGE_TARNAME],   [openvswitch])' && \
  echo 'm4_define([AT_PACKAGE_VERSION],   [2.11.90])' && \
  echo 'm4_define([AT_PACKAGE_STRING],    [openvswitch 2.11.90])' && \
  echo 'm4_define([AT_PACKAGE_BUGREPORT], [bugs@openvswitch.org])'; \
} >'./package.m4'
/bin/sh /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing autom4te --language=autotest -I '.' -o tests/testsuite.tmp tests/testsuite.at
mv tests/testsuite.tmp tests/testsuite
/bin/sh /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing autom4te --language=autotest -I '.' -o tests/system-kmod-testsuite.tmp tests/system-kmod-testsuite.at
/usr/bin/m4:tests/system-kmod-testsuite.at:22: cannot open `tests/ovn-macros.at': No such file or directory
autom4te: /usr/bin/m4 failed with exit status: 1
make[2]: *** [tests/system-kmod-testsuite] Error 1
make[2]: Leaving directory `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
make: *** [all] Error 2


Please check this out.  If you feel there has been an error, please email aconole@bytheb.org

Thanks,
0-day Robot
Darrell Ball May 28, 2019, 6:18 p.m. UTC | #2
On Sat, May 25, 2019 at 2:19 PM 0-day Robot <robot@bytheb.org> wrote:

> Bleep bloop.  Greetings Darrell Ball, I am a robot and I have tried out
> your patch.
> Thanks for your contribution.
>
> I encountered some error that I wasn't expecting.  See the details below.
>
>
> build:
>   sysconfdir='/usr/local/etc' \
>   bindir='/usr/local/bin' \
>   sbindir='/usr/local/sbin'
> mv northd/ovn-northd.8.tmp northd/ovn-northd.8
> printf '\043 Generated automatically -- do not modify!    -*-
> buffer-read-only: t -*-\n' > version.py.tmp && \
> echo 'VERSION = "2.11.90"' >> version.py.tmp && \
> if cmp -s version.py.tmp python/ovs/version.py; then touch
> python/ovs/version.py; rm version.py.tmp; else mv version.py.tmp
> python/ovs/version.py; fi
> :;{ \
>   echo '# Signature of the current package.' && \
>   echo 'm4_define([AT_PACKAGE_NAME],      [openvswitch])' && \
>   echo 'm4_define([AT_PACKAGE_TARNAME],   [openvswitch])' && \
>   echo 'm4_define([AT_PACKAGE_VERSION],   [2.11.90])' && \
>   echo 'm4_define([AT_PACKAGE_STRING],    [openvswitch 2.11.90])' && \
>   echo 'm4_define([AT_PACKAGE_BUGREPORT], [bugs@openvswitch.org])'; \
> } >'./package.m4'
> /bin/sh
> /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing
> autom4te --language=autotest -I '.' -o tests/testsuite.tmp tests/
> testsuite.at
> mv tests/testsuite.tmp tests/testsuite
> /bin/sh
> /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing
> autom4te --language=autotest -I '.' -o tests/system-kmod-testsuite.tmp
> tests/system-kmod-testsuite.at
> /usr/bin/m4:tests/system-kmod-testsuite.at:22: cannot open `tests/
> ovn-macros.at': No such file or directory
> autom4te: /usr/bin/m4 failed with exit status: 1
> make[2]: *** [tests/system-kmod-testsuite] Error 1
> make[2]: Leaving directory
> `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory
> `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
> make: *** [all] Error 2
>
>
>
I did not find issues in any testing, so not sure why the bot is failing.




> Please check this out.  If you feel there has been an error, please email
> aconole@bytheb.org
>
> Thanks,
> 0-day Robot
>
Aaron Conole May 28, 2019, 7:20 p.m. UTC | #3
Darrell Ball <dlu998@gmail.com> writes:

> On Sat, May 25, 2019 at 2:19 PM 0-day Robot <robot@bytheb.org> wrote:
>
>  Bleep bloop.  Greetings Darrell Ball, I am a robot and I have tried out your patch.
>  Thanks for your contribution.
>
>  I encountered some error that I wasn't expecting.  See the details below.
>
>  build:
>    sysconfdir='/usr/local/etc' \
>    bindir='/usr/local/bin' \
>    sbindir='/usr/local/sbin'
>  mv northd/ovn-northd.8.tmp northd/ovn-northd.8
>  printf '\043 Generated automatically -- do not modify!    -*- buffer-read-only: t -*-\n' > version.py.tmp && \
>  echo 'VERSION = "2.11.90"' >> version.py.tmp && \
>  if cmp -s version.py.tmp python/ovs/version.py; then touch python/ovs/version.py; rm version.py.tmp; else
>  mv version.py.tmp python/ovs/version.py; fi
>  :;{ \
>    echo '# Signature of the current package.' && \
>    echo 'm4_define([AT_PACKAGE_NAME],      [openvswitch])' && \
>    echo 'm4_define([AT_PACKAGE_TARNAME],   [openvswitch])' && \
>    echo 'm4_define([AT_PACKAGE_VERSION],   [2.11.90])' && \
>    echo 'm4_define([AT_PACKAGE_STRING],    [openvswitch 2.11.90])' && \
>    echo 'm4_define([AT_PACKAGE_BUGREPORT], [bugs@openvswitch.org])'; \
>  } >'./package.m4'
>  /bin/sh /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing autom4te
>  --language=autotest -I '.' -o tests/testsuite.tmp tests/testsuite.at
>  mv tests/testsuite.tmp tests/testsuite
>  /bin/sh /var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/build-aux/missing autom4te
>  --language=autotest -I '.' -o tests/system-kmod-testsuite.tmp tests/system-kmod-testsuite.at
>  /usr/bin/m4:tests/system-kmod-testsuite.at:22: cannot open `tests/ovn-macros.at': No such file or
>  directory
>  autom4te: /usr/bin/m4 failed with exit status: 1
>  make[2]: *** [tests/system-kmod-testsuite] Error 1
>  make[2]: Leaving directory `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
>  make[1]: *** [all-recursive] Error 1
>  make[1]: Leaving directory `/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN'
>  make: *** [all] Error 2
>
> I did not find issues in any testing, so not sure why the bot is failing.

This happened because the bot had an invalid regex for detecting OVN
patches.  I think I've fixed it now.  Sorry for the noise.

>  
>  Please check this out.  If you feel there has been an error, please email aconole@bytheb.org
>
>  Thanks,
>  0-day Robot
Gregory Rose May 30, 2019, 5:48 p.m. UTC | #4
On 5/25/2019 1:49 PM, Darrell Ball wrote:
> ovn-macros are needed to run the OVN system tests.
>
> Signed-off-by: Darrell Ball <dlu998@gmail.com>
> ---
>   tests/system-kmod-testsuite.at      | 1 +
>   tests/system-userspace-testsuite.at | 1 +
>   2 files changed, 2 insertions(+)
>
> diff --git a/tests/system-kmod-testsuite.at b/tests/system-kmod-testsuite.at
> index bda314a..2fe2e8f 100644
> --- a/tests/system-kmod-testsuite.at
> +++ b/tests/system-kmod-testsuite.at
> @@ -19,6 +19,7 @@ m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
>   m4_include([tests/ovs-macros.at])
>   m4_include([tests/ovsdb-macros.at])
>   m4_include([tests/ofproto-macros.at])
> +m4_include([tests/ovn-macros.at])
>   m4_include([tests/system-common-macros.at])
>   m4_include([tests/system-kmod-macros.at])
>   
> diff --git a/tests/system-userspace-testsuite.at b/tests/system-userspace-testsuite.at
> index c0aaa60..4af36be 100644
> --- a/tests/system-userspace-testsuite.at
> +++ b/tests/system-userspace-testsuite.at
> @@ -19,6 +19,7 @@ m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
>   m4_include([tests/ovs-macros.at])
>   m4_include([tests/ovsdb-macros.at])
>   m4_include([tests/ofproto-macros.at])
> +m4_include([tests/ovn-macros.at])
>   m4_include([tests/system-userspace-macros.at])
>   m4_include([tests/system-common-macros.at])
>   

I was wondering what had happened to all the OVN tests on my automated 
daily check-kmod jobs and just as I was about to start asking I see your 
patch.  Good timing.

Thanks!

LGTM...

Tested-by: Greg Rose <gvrose8192@gmail.com>
Reviewed-by: Greg Rose <gvrose8192@gmail.com>
Ilya Maximets June 3, 2019, 7:48 a.m. UTC | #5
On 25.05.2019 23:49, Darrell Ball wrote:
> ovn-macros are needed to run the OVN system tests.
> 
> Signed-off-by: Darrell Ball <dlu998@gmail.com>
> Tested-by: Greg Rose <gvrose8192@gmail.com>
> Reviewed-by: Greg Rose <gvrose8192@gmail.com>
> ---

Hi Darrell.
Thanks for the fix. It looks good.

BTW, I have a question. Are you able to run 'make check-system-userspace'
successfully? I have a lot of failures beside the OVN related ones.
I didn't look closely yet.

Best regards, Ilya Maximets.
Darrell Ball June 3, 2019, 3:55 p.m. UTC | #6
On Mon, Jun 3, 2019 at 12:48 AM Ilya Maximets <i.maximets@samsung.com>
wrote:

> On 25.05.2019 23:49, Darrell Ball wrote:
> > ovn-macros are needed to run the OVN system tests.
> >
> > Signed-off-by: Darrell Ball <dlu998@gmail.com>
> > Tested-by: Greg Rose <gvrose8192@gmail.com>
> > Reviewed-by: Greg Rose <gvrose8192@gmail.com>
> > ---
>
> Hi Darrell.
> Thanks for the fix. It looks good.
>
> BTW, I have a question. Are you able to run 'make check-system-userspace'
> successfully? I have a lot of failures beside the OVN related ones.
> I didn't look closely yet.
>

which tests are failing for you ?; I can check later.



>
> Best regards, Ilya Maximets.
>
Ilya Maximets June 3, 2019, 5:27 p.m. UTC | #7
On 03.06.2019 18:55, Darrell Ball wrote:
> 
> 
> On Mon, Jun 3, 2019 at 12:48 AM Ilya Maximets <i.maximets@samsung.com <mailto:i.maximets@samsung.com>> wrote:
> 
>     On 25.05.2019 23:49, Darrell Ball wrote:
>     > ovn-macros are needed to run the OVN system tests.
>     >
>     > Signed-off-by: Darrell Ball <dlu998@gmail.com <mailto:dlu998@gmail.com>>
>     > Tested-by: Greg Rose <gvrose8192@gmail.com <mailto:gvrose8192@gmail.com>>
>     > Reviewed-by: Greg Rose <gvrose8192@gmail.com <mailto:gvrose8192@gmail.com>>
>     > ---
> 
>     Hi Darrell.
>     Thanks for the fix. It looks good.
> 
>     BTW, I have a question. Are you able to run 'make check-system-userspace'
>     successfully? I have a lot of failures beside the OVN related ones.
>     I didn't look closely yet.
> 
> 
> which tests are failing for you ?; I can check later.

Thanks.
Here is the list of currently failing tests:

datapath-sanity

  9: datapath - ping over vxlan tunnel               FAILED (system-traffic.at:246)
 10: datapath - ping over vxlan6 tunnel              FAILED (system-traffic.at:289)
 16: datapath - ping over geneve tunnel              FAILED (system-traffic.at:522)
 17: datapath - flow resume with geneve tun_metadata FAILED (system-traffic.at:568)
 18: datapath - ping over geneve6 tunnel             FAILED (system-traffic.at:606)

nsh-datapath

121: nsh - decap header                              FAILED (system-traffic.at:5839)
122: nsh - replace header                            FAILED (system-traffic.at:5876)
123: nsh - forward                                   FAILED (system-traffic.at:5931)

layer3-tunnels

124: layer3 - ping over VXLAN-GPE                    FAILED (ovs-macros.at:219)
125: layer3 - ping over GRE                          FAILED (ovs-macros.at:219)


You may found a bit stripped version of system-userspace-testsuite.log here:
    https://pastebin.com/RX58PnKR

Best regards, Ilya Maximets.
Ilya Maximets June 3, 2019, 5:31 p.m. UTC | #8
On 25.05.2019 23:49, Darrell Ball wrote:
> ovn-macros are needed to run the OVN system tests.
> 
> Signed-off-by: Darrell Ball <dlu998@gmail.com>
> Tested-by: Greg Rose <gvrose8192@gmail.com>
> Reviewed-by: Greg Rose <gvrose8192@gmail.com>
> ---
>  tests/system-kmod-testsuite.at      | 1 +
>  tests/system-userspace-testsuite.at | 1 +
>  2 files changed, 2 insertions(+)
> 


Thanks, Darrell and Gregory!
Applied to master.

Best regards, Ilya Maximets.
Darrell Ball June 4, 2019, 4:37 a.m. UTC | #9
On Mon, Jun 3, 2019 at 10:27 AM Ilya Maximets <i.maximets@samsung.com>
wrote:

> On 03.06.2019 18:55, Darrell Ball wrote:
> >
> >
> > On Mon, Jun 3, 2019 at 12:48 AM Ilya Maximets <i.maximets@samsung.com
> <mailto:i.maximets@samsung.com>> wrote:
> >
> >     On 25.05.2019 23:49, Darrell Ball wrote:
> >     > ovn-macros are needed to run the OVN system tests.
> >     >
> >     > Signed-off-by: Darrell Ball <dlu998@gmail.com <mailto:
> dlu998@gmail.com>>
> >     > Tested-by: Greg Rose <gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com>>
> >     > Reviewed-by: Greg Rose <gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com>>
> >     > ---
> >
> >     Hi Darrell.
> >     Thanks for the fix. It looks good.
> >
> >     BTW, I have a question. Are you able to run 'make
> check-system-userspace'
> >     successfully? I have a lot of failures beside the OVN related ones.
> >     I didn't look closely yet.
> >
> >
> > which tests are failing for you ?; I can check later.
>
> Thanks.
> Here is the list of currently failing tests:
>

These tests are un-loved for some time
Copying couple folks.


>
> datapath-sanity
>
>   9: datapath - ping over vxlan tunnel               FAILED (
> system-traffic.at:246)
>  10: datapath - ping over vxlan6 tunnel              FAILED (
> system-traffic.at:289)
>  16: datapath - ping over geneve tunnel              FAILED (
> system-traffic.at:522)
>  17: datapath - flow resume with geneve tun_metadata FAILED (
> system-traffic.at:568)
>  18: datapath - ping over geneve6 tunnel             FAILED (
> system-traffic.at:606)
>

These should be bypassed for userspace due to dependencies on kernel modules
Also depends on routing table info - see below.


>
> nsh-datapath
>
> 121: nsh - decap header                              FAILED (
> system-traffic.at:5839)
> 122: nsh - replace header                            FAILED (
> system-traffic.at:5876)
> 123: nsh - forward                                   FAILED (
> system-traffic.at:5931)
>

+2019-06-04T03:38:14.365Z|00001|odp_util(revalidator5)|WARN|OVS_NSH_KEY_ATTR_MD1
present but declared mdtype 0 is not 1 (NSH_M_TYPE1)
+2019-06-04T03:38:14.366Z|00002|odp_util(revalidator5)|WARN|the flow mask
in error is:
skb_priority(0),tunnel(ttl=0,flags(0)),skb_mark(0),ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),recirc_id(0xffffffff),dp_hash(0),in_port(4294967295),packet_type(ns=65535,id=0xffff),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00),eth_type(0xffff),nsh(flags=0ttl=0,mdtype=0,np=255,spi=0xffffff,si=255),
for the following flow key:
in_port=2,vlan_tci=0x0000,dl_src=f2:ff:00:00:00:01,dl_dst=f2:ff:00:00:00:02,dl_type=0x894f,nsh_flags=0,nsh_ttl=8,nsh_mdtype=1,nsh_np=3,nsh_spi=0x100,nsh_si=3,nsh_c1=0x1020304,nsh_c2=0x5060708,nsh_c3=0x90a0b0c,nsh_c4=0xd0e0f10,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0

'OVS_NSH_KEY_ATTR_MD1 present but declared mdtype 0 is not 1 (NSH_M_TYPE1)'
also needs bisecting


>
> layer3-tunnels
>
> 124: layer3 - ping over VXLAN-GPE                    FAILED (
> ovs-macros.at:219)
> 125: layer3 - ping over GRE                          FAILED (
> ovs-macros.at:219)
>

First issue seems to be related to:
898d7b052       Ben Pfaff       Sat Mar 31 17:12:55 2018 -0700
 ovs-vswitchd: Do not use system routing table with --disable-system.
which adds suppressing routing table info/updates to 'disable-system'
directive, which system userspace tests use.

At least a couple ways to fix this first issue, including possibly more
selective disabling
If there are other non-system tests covering functionality, then possibly
just bypass.


>
> You may found a bit stripped version of system-userspace-testsuite.log
> here:
>     https://pastebin.com/RX58PnKR
>
> Best regards, Ilya Maximets.
>
Ilya Maximets June 4, 2019, 3:44 p.m. UTC | #10
On 04.06.2019 7:37, Darrell Ball wrote:
> 
> 
> On Mon, Jun 3, 2019 at 10:27 AM Ilya Maximets <i.maximets@samsung.com <mailto:i.maximets@samsung.com>> wrote:
> 
>     On 03.06.2019 18:55, Darrell Ball wrote:
>     >
>     >
>     > On Mon, Jun 3, 2019 at 12:48 AM Ilya Maximets <i.maximets@samsung.com <mailto:i.maximets@samsung.com> <mailto:i.maximets@samsung.com <mailto:i.maximets@samsung.com>>> wrote:
>     >
>     >     On 25.05.2019 23:49, Darrell Ball wrote:
>     >     > ovn-macros are needed to run the OVN system tests.
>     >     >
>     >     > Signed-off-by: Darrell Ball <dlu998@gmail.com <mailto:dlu998@gmail.com> <mailto:dlu998@gmail.com <mailto:dlu998@gmail.com>>>
>     >     > Tested-by: Greg Rose <gvrose8192@gmail.com <mailto:gvrose8192@gmail.com> <mailto:gvrose8192@gmail.com <mailto:gvrose8192@gmail.com>>>
>     >     > Reviewed-by: Greg Rose <gvrose8192@gmail.com <mailto:gvrose8192@gmail.com> <mailto:gvrose8192@gmail.com <mailto:gvrose8192@gmail.com>>>
>     >     > ---
>     >
>     >     Hi Darrell.
>     >     Thanks for the fix. It looks good.
>     >
>     >     BTW, I have a question. Are you able to run 'make check-system-userspace'
>     >     successfully? I have a lot of failures beside the OVN related ones.
>     >     I didn't look closely yet.
>     >
>     >
>     > which tests are failing for you ?; I can check later.
> 
>     Thanks.
>     Here is the list of currently failing tests:
> 
> 
> These tests are un-loved for some time
> Copying couple folks.
>  
> 
> 
>     datapath-sanity
> 
>       9: datapath - ping over vxlan tunnel               FAILED (system-traffic.at:246 <https://protect2.fireeye.com/url?k=ca2351833f1c13f5.ca22dacc-2d077141969cf994&u=http://system-traffic.at:246>)
>      10: datapath - ping over vxlan6 tunnel              FAILED (system-traffic.at:289 <https://protect2.fireeye.com/url?k=806bcaddf0570229.806a4192-31aa7a98ff0ea74c&u=http://system-traffic.at:289>)
>      16: datapath - ping over geneve tunnel              FAILED (system-traffic.at:522 <https://protect2.fireeye.com/url?k=1b59b694f3eb247f.1b583ddb-0660c0ec5c47f5bf&u=http://system-traffic.at:522>)
>      17: datapath - flow resume with geneve tun_metadata FAILED (system-traffic.at:568 <https://protect2.fireeye.com/url?k=a4cd38cbf817df78.a4ccb384-2d47344d2cec23cb&u=http://system-traffic.at:568>)
>      18: datapath - ping over geneve6 tunnel             FAILED (system-traffic.at:606 <https://protect2.fireeye.com/url?k=e826024a4d9e8b12.e8278905-9b148a9e2dd10697&u=http://system-traffic.at:606>)
> 
> 
> These should be bypassed for userspace due to dependencies on kernel modules
> Also depends on routing table info - see below.
>  
> 
> 
>     nsh-datapath
> 
>     121: nsh - decap header                              FAILED (system-traffic.at:5839 <http://system-traffic.at:5839>)
>     122: nsh - replace header                            FAILED (system-traffic.at:5876 <http://system-traffic.at:5876>)
>     123: nsh - forward                                   FAILED (system-traffic.at:5931 <http://system-traffic.at:5931>)
> 
> 
> +2019-06-04T03:38:14.365Z|00001|odp_util(revalidator5)|WARN|OVS_NSH_KEY_ATTR_MD1 present but declared mdtype 0 is not 1 (NSH_M_TYPE1)
> +2019-06-04T03:38:14.366Z|00002|odp_util(revalidator5)|WARN|the flow mask in error is: skb_priority(0),tunnel(ttl=0,flags(0)),skb_mark(0),ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),recirc_id(0xffffffff),dp_hash(0),in_port(4294967295),packet_type(ns=65535,id=0xffff),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00),eth_type(0xffff),nsh(flags=0ttl=0,mdtype=0,np=255,spi=0xffffff,si=255), for the following flow key: in_port=2,vlan_tci=0x0000,dl_src=f2:ff:00:00:00:01,dl_dst=f2:ff:00:00:00:02,dl_type=0x894f,nsh_flags=0,nsh_ttl=8,nsh_mdtype=1,nsh_np=3,nsh_spi=0x100,nsh_si=3,nsh_c1=0x1020304,nsh_c2=0x5060708,nsh_c3=0x90a0b0c,nsh_c4=0xd0e0f10,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
> 
> 'OVS_NSH_KEY_ATTR_MD1 present but declared mdtype 0 is not 1 (NSH_M_TYPE1)'
> also needs bisecting
>  
> 
> 
>     layer3-tunnels
> 
>     124: layer3 - ping over VXLAN-GPE                    FAILED (ovs-macros.at:219 <http://ovs-macros.at:219>)
>     125: layer3 - ping over GRE                          FAILED (ovs-macros.at:219 <http://ovs-macros.at:219>)
> 
> 
> First issue seems to be related to:
> 898d7b052       Ben Pfaff       Sat Mar 31 17:12:55 2018 -0700  ovs-vswitchd: Do not use system routing table with --disable-system.
> which adds suppressing routing table info/updates to 'disable-system' directive, which system userspace tests use.

Good catch. Thank you!
By reverting this patch all the tests passed except two NSH tests (122 and 123).
Actually, userspace tunneling usually relates on system routing tables. So, it's
unclear what to do with this change.

> 
> At least a couple ways to fix this first issue, including possibly more selective disabling
> If there are other non-system tests covering functionality, then possibly just bypass.
> 
> 
> 
>     You may found a bit stripped version of system-userspace-testsuite.log here:
>         https://pastebin.com/RX58PnKR
> 
>     Best regards, Ilya Maximets.
>
Darrell Ball June 4, 2019, 5:45 p.m. UTC | #11
On Tue, Jun 4, 2019 at 8:44 AM Ilya Maximets <i.maximets@samsung.com> wrote:

> On 04.06.2019 7:37, Darrell Ball wrote:
> >
> >
> > On Mon, Jun 3, 2019 at 10:27 AM Ilya Maximets <i.maximets@samsung.com
> <mailto:i.maximets@samsung.com>> wrote:
> >
> >     On 03.06.2019 18:55, Darrell Ball wrote:
> >     >
> >     >
> >     > On Mon, Jun 3, 2019 at 12:48 AM Ilya Maximets <
> i.maximets@samsung.com <mailto:i.maximets@samsung.com> <mailto:
> i.maximets@samsung.com <mailto:i.maximets@samsung.com>>> wrote:
> >     >
> >     >     On 25.05.2019 23:49, Darrell Ball wrote:
> >     >     > ovn-macros are needed to run the OVN system tests.
> >     >     >
> >     >     > Signed-off-by: Darrell Ball <dlu998@gmail.com <mailto:
> dlu998@gmail.com> <mailto:dlu998@gmail.com <mailto:dlu998@gmail.com>>>
> >     >     > Tested-by: Greg Rose <gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com> <mailto:gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com>>>
> >     >     > Reviewed-by: Greg Rose <gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com> <mailto:gvrose8192@gmail.com <mailto:
> gvrose8192@gmail.com>>>
> >     >     > ---
> >     >
> >     >     Hi Darrell.
> >     >     Thanks for the fix. It looks good.
> >     >
> >     >     BTW, I have a question. Are you able to run 'make
> check-system-userspace'
> >     >     successfully? I have a lot of failures beside the OVN related
> ones.
> >     >     I didn't look closely yet.
> >     >
> >     >
> >     > which tests are failing for you ?; I can check later.
> >
> >     Thanks.
> >     Here is the list of currently failing tests:
> >
> >
> > These tests are un-loved for some time
> > Copying couple folks.
> >
> >
> >
> >     datapath-sanity
> >
> >       9: datapath - ping over vxlan tunnel               FAILED (
> system-traffic.at:246 <
> https://protect2.fireeye.com/url?k=ca2351833f1c13f5.ca22dacc-2d077141969cf994&u=http://system-traffic.at:246
> >)
> >      10: datapath - ping over vxlan6 tunnel              FAILED (
> system-traffic.at:289 <
> https://protect2.fireeye.com/url?k=806bcaddf0570229.806a4192-31aa7a98ff0ea74c&u=http://system-traffic.at:289
> >)
> >      16: datapath - ping over geneve tunnel              FAILED (
> system-traffic.at:522 <
> https://protect2.fireeye.com/url?k=1b59b694f3eb247f.1b583ddb-0660c0ec5c47f5bf&u=http://system-traffic.at:522
> >)
> >      17: datapath - flow resume with geneve tun_metadata FAILED (
> system-traffic.at:568 <
> https://protect2.fireeye.com/url?k=a4cd38cbf817df78.a4ccb384-2d47344d2cec23cb&u=http://system-traffic.at:568
> >)
> >      18: datapath - ping over geneve6 tunnel             FAILED (
> system-traffic.at:606 <
> https://protect2.fireeye.com/url?k=e826024a4d9e8b12.e8278905-9b148a9e2dd10697&u=http://system-traffic.at:606
> >)
> >
> >
> > These should be bypassed for userspace due to dependencies on kernel
> modules
> > Also depends on routing table info - see below.
> >
> >
> >
> >     nsh-datapath
> >
> >     121: nsh - decap header                              FAILED (
> system-traffic.at:5839 <http://system-traffic.at:5839>)
> >     122: nsh - replace header                            FAILED (
> system-traffic.at:5876 <http://system-traffic.at:5876>)
> >     123: nsh - forward                                   FAILED (
> system-traffic.at:5931 <http://system-traffic.at:5931>)
> >
> >
> >
> +2019-06-04T03:38:14.365Z|00001|odp_util(revalidator5)|WARN|OVS_NSH_KEY_ATTR_MD1
> present but declared mdtype 0 is not 1 (NSH_M_TYPE1)
> > +2019-06-04T03:38:14.366Z|00002|odp_util(revalidator5)|WARN|the flow
> mask in error is:
> skb_priority(0),tunnel(ttl=0,flags(0)),skb_mark(0),ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),recirc_id(0xffffffff),dp_hash(0),in_port(4294967295),packet_type(ns=65535,id=0xffff),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00),eth_type(0xffff),nsh(flags=0ttl=0,mdtype=0,np=255,spi=0xffffff,si=255),
> for the following flow key:
> in_port=2,vlan_tci=0x0000,dl_src=f2:ff:00:00:00:01,dl_dst=f2:ff:00:00:00:02,dl_type=0x894f,nsh_flags=0,nsh_ttl=8,nsh_mdtype=1,nsh_np=3,nsh_spi=0x100,nsh_si=3,nsh_c1=0x1020304,nsh_c2=0x5060708,nsh_c3=0x90a0b0c,nsh_c4=0xd0e0f10,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
> >
> > 'OVS_NSH_KEY_ATTR_MD1 present but declared mdtype 0 is not 1
> (NSH_M_TYPE1)'
> > also needs bisecting
> >
> >
> >
> >     layer3-tunnels
> >
> >     124: layer3 - ping over VXLAN-GPE                    FAILED (
> ovs-macros.at:219 <http://ovs-macros.at:219>)
> >     125: layer3 - ping over GRE                          FAILED (
> ovs-macros.at:219 <http://ovs-macros.at:219>)
> >
> >
> > First issue seems to be related to:
> > 898d7b052       Ben Pfaff       Sat Mar 31 17:12:55 2018 -0700
>  ovs-vswitchd: Do not use system routing table with --disable-system.
> > which adds suppressing routing table info/updates to 'disable-system'
> directive, which system userspace tests use.
>
> Good catch. Thank you!
> By reverting this patch all the tests passed except two NSH tests (122 and
> 123).
> Actually, userspace tunneling usually relates on system routing tables.
> So, it's
> unclear what to do with this change.
>

1/ The NSH test failures are unrelated to the above change
     They still fail for the same reasons for me when I reverted the above
change, which I would expect.

nsh-datapath

121: nsh - decap header                              FAILED (
system-traffic.at:5839)
122: nsh - replace header                            FAILED (
system-traffic.at:5876)
123: nsh - forward                                   FAILED (
system-traffic.at:5931)


2/ In terms of whether to revert the change, we may consider "selective
disabling" in tests w.r.t. the system routing table
   per the comment below.

3/ Unfortunately, I get other warn logs that fail the tests.


>
> >
> > At least a couple ways to fix this first issue, including possibly more
> selective disabling

> If there are other non-system tests covering functionality, then possibly
> just bypass.
> >
> >
> >
> >     You may found a bit stripped version of
> system-userspace-testsuite.log here:
> >         https://pastebin.com/RX58PnKR
> >
> >     Best regards, Ilya Maximets.
> >
>

Patch
diff mbox series

diff --git a/tests/system-kmod-testsuite.at b/tests/system-kmod-testsuite.at
index bda314a..2fe2e8f 100644
--- a/tests/system-kmod-testsuite.at
+++ b/tests/system-kmod-testsuite.at
@@ -19,6 +19,7 @@  m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
 m4_include([tests/ovs-macros.at])
 m4_include([tests/ovsdb-macros.at])
 m4_include([tests/ofproto-macros.at])
+m4_include([tests/ovn-macros.at])
 m4_include([tests/system-common-macros.at])
 m4_include([tests/system-kmod-macros.at])
 
diff --git a/tests/system-userspace-testsuite.at b/tests/system-userspace-testsuite.at
index c0aaa60..4af36be 100644
--- a/tests/system-userspace-testsuite.at
+++ b/tests/system-userspace-testsuite.at
@@ -19,6 +19,7 @@  m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
 m4_include([tests/ovs-macros.at])
 m4_include([tests/ovsdb-macros.at])
 m4_include([tests/ofproto-macros.at])
+m4_include([tests/ovn-macros.at])
 m4_include([tests/system-userspace-macros.at])
 m4_include([tests/system-common-macros.at])