| Message ID | 20210218195112.1391467-1-blp@ovn.org |
|---|---|
| Headers | show |
| Series | Add DDlog implementation of ovn-northd | expand |
This passed in the ovsrobot CI: https://github.com/ovsrobot/ovn/actions/runs/579307688 I think that we have arrived at consensus to push this to ovn master after branching happens tomorrow. Please, let me know if I misunderstood!
On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > This passed in the ovsrobot CI: > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > I think that we have arrived at consensus to push this to ovn master > after branching happens tomorrow. Please, let me know if I > misunderstood! I think understanding is correct. I think Mark needs to create the branch. Thanks Numan > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > This passed in the ovsrobot CI: > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > I think that we have arrived at consensus to push this to ovn master > > after branching happens tomorrow. Please, let me know if I > > misunderstood! > > I think understanding is correct. I think Mark needs to create the branch. Hi Ben, I am trying to compile with ddlog configured, but the compilation is failing for me for some reason. Can you please help me with this ? I tried both on Fedora 33 and also on ubuntu. Below is the log ---- vagrant@vagrant:~/ovs/ovn$ make cd . && /bin/bash /home/vagrant/ovs/ovn/build-aux/missing automake-1.16 --foreign Makefile cd . && /bin/bash ./config.status Makefile depfiles config.status: creating Makefile config.status: executing depfiles commands PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in annotate ./ovn-sb.ovsschema ./lib/ovn-sb-idl.ann > lib/ovn-sb-idl.ovsidl.tmp && \ mv lib/ovn-sb-idl.ovsidl.tmp lib/ovn-sb-idl.ovsidl PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in c-idl-source lib/ovn-sb-idl.ovsidl > lib/ovn-sb-idl.c.tmp && mv lib/ovn-sb-idl.c.tmp lib/ovn-sb-idl.c PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in c-idl-header lib/ovn-sb-idl.ovsidl > lib/ovn-sb-idl.h.tmp && mv lib/ovn-sb-idl.h.tmp lib/ovn-sb-idl.h PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 ./northd/ovsdb2ddlog2c -p sb_ -f ovn-sb.ovsschema --output-file northd/ovn-northd-ddlog-sb.inc $(cat ./northd/ovn-sb.dlopts) PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 ./northd/ovsdb2ddlog2c -p nb_ -f ovn-nb.ovsschema --output-file northd/ovn-northd-ddlog-nb.inc $(cat ./northd/ovn-nb.dlopts) ddlog -i northd/ovn_northd.dl -o ./northd -L /home/vagrant/.local/ddlog/lib -L ./northd error: module 'OVN_Northbound' imported by northd/ovn_northd.dl not found. Paths searched: /home/vagrant/ovs/ovn/northd/OVN_Northbound.dl /home/vagrant/.local/ddlog/lib/OVN_Northbound.dl ----- For some reason the below target define in northd/automake.mk is not getting invoked ---- northd/OVN_Northbound.dl: ovn-nb.ovsschema northd/ovn-nb.dlopts $(AM_V_GEN)ovsdb2ddlog -f $< --output-file $@ $(nb_opts) ---- FYI - The branch-21.03 is now created and I think we can accept these patches. Thanks Numan > > Thanks > Numan > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > >
On Tue, Feb 23, 2021 at 7:38 PM Numan Siddique <numans@ovn.org> wrote: > > On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > > > This passed in the ovsrobot CI: > > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > > > I think that we have arrived at consensus to push this to ovn master > > > after branching happens tomorrow. Please, let me know if I > > > misunderstood! > > > > I think understanding is correct. I think Mark needs to create the branch. > > Hi Ben, > > I am trying to compile with ddlog configured, but the compilation is > failing for me > for some reason. Can you please help me with this ? > > I tried both on Fedora 33 and also on ubuntu. > > Below is the log > > ---- > vagrant@vagrant:~/ovs/ovn$ make > cd . && /bin/bash /home/vagrant/ovs/ovn/build-aux/missing > automake-1.16 --foreign Makefile > cd . && /bin/bash ./config.status Makefile depfiles > config.status: creating Makefile > config.status: executing depfiles commands > PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH > PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 > /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in annotate > ./ovn-sb.ovsschema ./lib/ovn-sb-idl.ann > lib/ovn-sb-idl.ovsidl.tmp && > \ > mv lib/ovn-sb-idl.ovsidl.tmp lib/ovn-sb-idl.ovsidl > PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH > PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 > /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in c-idl-source > lib/ovn-sb-idl.ovsidl > lib/ovn-sb-idl.c.tmp && mv > lib/ovn-sb-idl.c.tmp lib/ovn-sb-idl.c > PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH > PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 > /home/vagrant/ovs/ovn/ovs/ovsdb/ovsdb-idlc.in c-idl-header > lib/ovn-sb-idl.ovsidl > lib/ovn-sb-idl.h.tmp && mv > lib/ovn-sb-idl.h.tmp lib/ovn-sb-idl.h > PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH > PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 ./northd/ovsdb2ddlog2c -p > sb_ -f ovn-sb.ovsschema --output-file northd/ovn-northd-ddlog-sb.inc > $(cat ./northd/ovn-sb.dlopts) > PYTHONPATH=/home/vagrant/ovs/ovn/ovs/python":"$PYTHONPATH > PYTHONDONTWRITEBYTECODE=yes /usr/bin/python3 ./northd/ovsdb2ddlog2c -p > nb_ -f ovn-nb.ovsschema --output-file northd/ovn-northd-ddlog-nb.inc > $(cat ./northd/ovn-nb.dlopts) > ddlog -i northd/ovn_northd.dl -o ./northd -L > /home/vagrant/.local/ddlog/lib -L ./northd > > error: module 'OVN_Northbound' imported by northd/ovn_northd.dl not > found. Paths searched: > /home/vagrant/ovs/ovn/northd/OVN_Northbound.dl > /home/vagrant/.local/ddlog/lib/OVN_Northbound.dl > ----- > > For some reason the below target define in northd/automake.mk is not > getting invoked > ---- > northd/OVN_Northbound.dl: ovn-nb.ovsschema northd/ovn-nb.dlopts > $(AM_V_GEN)ovsdb2ddlog -f $< --output-file $@ $(nb_opts) > ---- > I worked around the problem by manually running -- nb_opts=$(cat northd/ovn-nb.dlopts) ovsdb2ddlog -f ovn-nb.ovsschema --output-file northd/OVN_Northbound.dl $nb_opts sb_opts=$(cat northd/ovn-sb.dlopts) ovsdb2ddlog -f ovn-sb.ovsschema --output-file northd/OVN_Soutbound.dl $sb_opts --- Thanks Numan > FYI - The branch-21.03 is now created and I think we can accept these patches. > > Thanks > Numan > > > > > Thanks > > Numan > > > > > _______________________________________________ > > > dev mailing list > > > dev@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > >
On Tue, Feb 23, 2021 at 07:38:12PM +0530, Numan Siddique wrote: > On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > > > This passed in the ovsrobot CI: > > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > > > I think that we have arrived at consensus to push this to ovn master > > > after branching happens tomorrow. Please, let me know if I > > > misunderstood! > > > > I think understanding is correct. I think Mark needs to create the branch. > > Hi Ben, > > I am trying to compile with ddlog configured, but the compilation is > failing for me > for some reason. Can you please help me with this ? Oops. I made a dumb typo. It didn't show up in my testing because I guess a -j12 build typically built OVN_Northbound.dl and OVN_Southbound.dl early enough. Here's the fix: diff --git a/northd/automake.mk b/northd/automake.mk index 17c394c82082..d64d1d197c77 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -29,7 +29,7 @@ ddlog_sources = \ northd/ovn.dl \ northd/ovn.rs \ northd/helpers.dl -ddlog_nodist_source = \ +ddlog_nodist_sources = \ northd/OVN_Northbound.dl \ northd/OVN_Southbound.dl
On Tue, Feb 23, 2021 at 11:16 PM Ben Pfaff <blp@ovn.org> wrote: > > On Tue, Feb 23, 2021 at 07:38:12PM +0530, Numan Siddique wrote: > > On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > > > > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > > > > > This passed in the ovsrobot CI: > > > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > > > > > I think that we have arrived at consensus to push this to ovn master > > > > after branching happens tomorrow. Please, let me know if I > > > > misunderstood! > > > > > > I think understanding is correct. I think Mark needs to create the branch. > > > > Hi Ben, > > > > I am trying to compile with ddlog configured, but the compilation is > > failing for me > > for some reason. Can you please help me with this ? > > Oops. I made a dumb typo. It didn't show up in my testing because I > guess a -j12 build typically built OVN_Northbound.dl and > OVN_Southbound.dl early enough. Here's the fix: > > diff --git a/northd/automake.mk b/northd/automake.mk > index 17c394c82082..d64d1d197c77 100644 > --- a/northd/automake.mk > +++ b/northd/automake.mk > @@ -29,7 +29,7 @@ ddlog_sources = \ > northd/ovn.dl \ > northd/ovn.rs \ > northd/helpers.dl > -ddlog_nodist_source = \ > +ddlog_nodist_sources = \ > northd/OVN_Northbound.dl \ > northd/OVN_Southbound.dl Thank you. It works. For the whole series Acked-by: Numan Siddique <numans@ovn.org> Unfortunately because of a couple of recent commits [1], you have to rebase the code and you may have to add some more code in northd-ddlog to sync up with c northd. [1] - https://github.com/ovn-org/ovn/commit/679d3550303ad1e5998e56bbcc63e9c4948522fc https://github.com/ovn-org/ovn/commit/dd94f1266ca4f3c750bc59c474ea342ef3ff9983 https://github.com/ovn-org/ovn/commit/f1f0de254157520a1bc754744e258eff9017a145 Thanks Numan > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
On Wed, Feb 24, 2021 at 12:40:17AM +0530, Numan Siddique wrote: > On Tue, Feb 23, 2021 at 11:16 PM Ben Pfaff <blp@ovn.org> wrote: > > > > On Tue, Feb 23, 2021 at 07:38:12PM +0530, Numan Siddique wrote: > > > On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > > > > > > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > > > > > > > This passed in the ovsrobot CI: > > > > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > > > > > > > I think that we have arrived at consensus to push this to ovn master > > > > > after branching happens tomorrow. Please, let me know if I > > > > > misunderstood! > > > > > > > > I think understanding is correct. I think Mark needs to create the branch. > > > > > > Hi Ben, > > > > > > I am trying to compile with ddlog configured, but the compilation is > > > failing for me > > > for some reason. Can you please help me with this ? > > > > Oops. I made a dumb typo. It didn't show up in my testing because I > > guess a -j12 build typically built OVN_Northbound.dl and > > OVN_Southbound.dl early enough. Here's the fix: > > > > diff --git a/northd/automake.mk b/northd/automake.mk > > index 17c394c82082..d64d1d197c77 100644 > > --- a/northd/automake.mk > > +++ b/northd/automake.mk > > @@ -29,7 +29,7 @@ ddlog_sources = \ > > northd/ovn.dl \ > > northd/ovn.rs \ > > northd/helpers.dl > > -ddlog_nodist_source = \ > > +ddlog_nodist_sources = \ > > northd/OVN_Northbound.dl \ > > northd/OVN_Southbound.dl > > Thank you. It works. > > For the whole series > Acked-by: Numan Siddique <numans@ovn.org> > > Unfortunately because of a couple of recent commits [1], you have to > rebase the code and you may have to add some more code in northd-ddlog to > sync up with c northd. Thanks so much. I rebased, added the missing features, tested, fixed my bugs, and applied this series to OVN master.
On Thu, Feb 25, 2021 at 12:35 PM Ben Pfaff <blp@ovn.org> wrote: > > On Wed, Feb 24, 2021 at 12:40:17AM +0530, Numan Siddique wrote: > > On Tue, Feb 23, 2021 at 11:16 PM Ben Pfaff <blp@ovn.org> wrote: > > > > > > On Tue, Feb 23, 2021 at 07:38:12PM +0530, Numan Siddique wrote: > > > > On Tue, Feb 23, 2021 at 4:10 PM Numan Siddique <numans@ovn.org> wrote: > > > > > > > > > > On Fri, Feb 19, 2021 at 4:10 AM Ben Pfaff <blp@ovn.org> wrote: > > > > > > > > > > > > This passed in the ovsrobot CI: > > > > > > https://github.com/ovsrobot/ovn/actions/runs/579307688 > > > > > > > > > > > > I think that we have arrived at consensus to push this to ovn master > > > > > > after branching happens tomorrow. Please, let me know if I > > > > > > misunderstood! > > > > > > > > > > I think understanding is correct. I think Mark needs to create the branch. > > > > > > > > Hi Ben, > > > > > > > > I am trying to compile with ddlog configured, but the compilation is > > > > failing for me > > > > for some reason. Can you please help me with this ? > > > > > > Oops. I made a dumb typo. It didn't show up in my testing because I > > > guess a -j12 build typically built OVN_Northbound.dl and > > > OVN_Southbound.dl early enough. Here's the fix: > > > > > > diff --git a/northd/automake.mk b/northd/automake.mk > > > index 17c394c82082..d64d1d197c77 100644 > > > --- a/northd/automake.mk > > > +++ b/northd/automake.mk > > > @@ -29,7 +29,7 @@ ddlog_sources = \ > > > northd/ovn.dl \ > > > northd/ovn.rs \ > > > northd/helpers.dl > > > -ddlog_nodist_source = \ > > > +ddlog_nodist_sources = \ > > > northd/OVN_Northbound.dl \ > > > northd/OVN_Southbound.dl > > > > Thank you. It works. > > > > For the whole series > > Acked-by: Numan Siddique <numans@ovn.org> > > > > Unfortunately because of a couple of recent commits [1], you have to > > rebase the code and you may have to add some more code in northd-ddlog to > > sync up with c northd. > > Thanks so much. I rebased, added the missing features, tested, fixed my > bugs, and applied this series to OVN master. Thanks to you, Leonid and Justin for adding the ddlog support. Numan > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
v10.1: - Fold in fix to northd makefile. v10: - Fix linking problem reported by Dumitru and others. - Update to ddlog 0.36.0 (thanks Leonid!). - Port OVN commits up to 858d1dd716db ("ofctrl: Fix the assert seen when flood removing flows.") - Fix IGMP and MLD snoop/querier/reply tests (thanks Dumitru!). - New commit "ovn-sb: Allow Multicast_Group to have empty set of ports." - New commit "ovn-northd: Simplify logic in build_bfd_table()." - New commit "tests: Wait for updates in "check BFD config propagation to BFD" test." v9: - Add ddlog.stamp to BUILT_SOURCES, to fix the dependencies "for sure". - Rebase against ovn/master. - Port commit fdf295d5eb3a ("pinctrl: Honor always_learn_from_arp_request for self created MAC_Bindings.") - Update to work with and require v2 of the ovsdb-cs library (see https://mail.openvswitch.org/pipermail/ovs-dev/2020-December/378180.html). - All non-skipped tests pass, although some are still racy. - This is also available as the ddlog14 branch here: https://github.com/blp/ovs-reviews/commits/ddlog14 - Haven't done performance tests. I aim to do that for v10. v8: - Rebase against latest ovn master. - Port commit f9cab11d5fab ("Allow explicit setting of the SNAT zone on a gateway router.") - Port commit f1119c125765 ("northd: Refactor load balancer vip parsing.") - Port commit 1dd27ea7aea4 ("Provide the option to pin ovn-controller and ovn-northd to a specific version."). - Port commit 880dca99eaf7 ("northd: Enhance the implementation of ACL log meters (pre-ddlog merge).") - Make ovn-northd-ddlog use the new ovsdb-cs (client sync) library proposed for OVS (https://mail.openvswitch.org/pipermail/ovs-dev/2020-November/377828.html). In addition to eliminating duplicated code, this should make it able to connect to a raft cluster. - Further improvements to test improvement patch (mostly added "check"s and AS_BOX(...) to announce parts of a test) - Modernize some of the ddlog code a bit, e.g. by using the method-like calls such as foo.get("key") in place of map_get(foo, "key"). v7: - Fixed "make dist" (thanks to Numan for reporting this.) - Fix probe interval setting (thanks to Numan again). - Small amount of improvement of OVN DDlog code (use lambdas in more places). - Fine-tuning of ovn-northd-ddlog code for jsonrpc session usage. - Fix newline issues in tests reported by Dumitru. - Add ack from Dumitru. v6: - Applied and dropped patch for "test a == b". - Applied and dropped patch to allow more args to wait_for_row_count(). - Applied and dropped ARP/ND broadcast test improvement patch. - Fix whitespace in patch to export ddlog_warn() and ddlog_err(). v5: - Drop patch for ACL log meters (this was included in v4.1 by mistake). - In addition to waiting for database, also wait for ports to come up. - New patch to support more arguments to wait_for_row_count that waiting for ports indirectly needed. - Port commit c29221d9322a ("Allow VLAN traffic when LS:vlan-passthru=true"). - Port commit 36a8745de859 ("Add support for DHCP options 28 (Broadcast Address)") - Add patch to improve "ARP/ND request broadcast limiting" test. v4: - Fix dependencies for parallel build. - Fix spelling error in documentation. - Use --wait=sb, not --wait=hv, when no chassis are running. - Fixed IGMP and MLD tests by porting commit 9d2e8d32fb98 ("ofctrl.c: Fix duplicated flow handling in I-P while merging opposite changes."), which I had missed before. Ben Pfaff (4): tests: Prepare for multiple northd types. tests: Wait for updates in "check BFD config propagation to BFD" test. ovn-northd: Simplify logic in build_bfd_table(). ovn-sb: Allow Multicast_Group to have empty set of ports. Leonid Ryzhyk (2): Export `VLOG_WARN` and `VLOG_ERR` from libovn for use in ddlog ovn-northd-ddlog: New implementation of ovn-northd based on ddlog. Documentation/automake.mk | 2 + Documentation/intro/install/general.rst | 31 +- Documentation/topics/debugging-ddlog.rst | 280 + Documentation/topics/index.rst | 1 + Documentation/tutorials/ddlog-new-feature.rst | 362 + Documentation/tutorials/index.rst | 1 + NEWS | 6 + TODO.rst | 6 + acinclude.m4 | 43 + build-aux/automake.mk | 1 + build-aux/libtool-deps | 35 + configure.ac | 5 + lib/ovn-util.c | 15 + lib/ovn-util.h | 5 + m4/ovn.m4 | 16 + northd/.gitignore | 4 + northd/automake.mk | 108 + northd/helpers.dl | 96 + northd/ipam.dl | 505 ++ northd/lrouter.dl | 761 ++ northd/lswitch.dl | 739 ++ northd/multicast.dl | 261 + northd/ovn-nb.dlopts | 15 + northd/ovn-northd-ddlog.c | 1286 +++ northd/ovn-northd.c | 20 +- northd/ovn-sb.dlopts | 31 + northd/ovn.dl | 383 + northd/ovn.rs | 858 ++ northd/ovn.toml | 2 + northd/ovn_northd.dl | 7820 +++++++++++++++++ northd/ovsdb2ddlog2c | 127 + ovn-sb.ovsschema | 6 +- tests/atlocal.in | 7 + tests/ovn-ic.at | 11 +- tests/ovn-macros.at | 99 +- tests/ovn-northd.at | 272 +- tests/ovn.at | 313 +- tests/ovs-macros.at | 23 +- tests/system-ovn.at | 124 +- tutorial/ovs-sandbox | 24 +- utilities/checkpatch.py | 2 +- utilities/ovn-ctl | 20 +- 42 files changed, 14542 insertions(+), 184 deletions(-) create mode 100644 Documentation/topics/debugging-ddlog.rst create mode 100644 Documentation/tutorials/ddlog-new-feature.rst create mode 100755 build-aux/libtool-deps create mode 100644 northd/helpers.dl create mode 100644 northd/ipam.dl create mode 100644 northd/lrouter.dl create mode 100644 northd/lswitch.dl create mode 100644 northd/multicast.dl create mode 100644 northd/ovn-nb.dlopts create mode 100644 northd/ovn-northd-ddlog.c create mode 100644 northd/ovn-sb.dlopts create mode 100644 northd/ovn.dl create mode 100644 northd/ovn.rs create mode 100644 northd/ovn.toml create mode 100644 northd/ovn_northd.dl create mode 100755 northd/ovsdb2ddlog2c