| Message ID | 20230513000356.2475960-1-hzhou@ovn.org |
|---|---|
| Headers | show |
| Series | ovn-northd incremental processing for VIF changes | expand |
On Sat, May 13, 2023 at 2:04 AM Han Zhou <hzhou@ovn.org> wrote: > This series implements incremental processing for VIF (regular logical > switch > ports of VM/pod) changes in ovn-northd. The performance gains of a single > logical switch port creation has improved significantly. > > Below are the performance test results simulating an ovn-k8s topology of > 500 > nodes x 50 lsp per node, with Intel i9-7920X CPU @ 2.90GHz: > > Before: > lsp-del: 955ms > lsp-add: 919ms > > After: > lsp-del: 776ms (~20% reduction) > lsp-add: 30ms (~97% reduction) > > Note: the test uses single thread of northd. > > In addition, the last 5 patches focuses on reducing the number of > recomputes > end-to-end for the VIF related changes when NB/SB update notifications come > back to northd. Before these patches, there were 5 recompute triggered > because > of the back-and-forth NB/SB updates, and after these changes there are zero > recomputes for the VIF creation + binding scenario. > > Some TODOs: > - For VIF deletion/update, the lflow node still recomputes. This will be > taken > care after implementing hash index on the lflows (needs to be > thread-safe and > efficient). This is why the lsp-del improvement is only 20%, and also the > reason why there is still one recompute for deletion/update. > - When there are continuous changes to NB/SB, there can be updates > received by > ovn-northd when it still has outstanding transactions, and it will > fallback > to recompute in this situation. This problem needs to be resolved because > large scale environment usually has high churn rates. > - Relax the constraint for LSP changes to allow for more incremental > processing. This can be gradually added according to the real needs. > > Han Zhou (14): > northd: Remove bfd_connections from northd_data. > northd: Enable persistence of logical flows between engine runs. > inc-proc-eng.h: Refactor I-P engine node macros. > ovn.at: Fix test case: delete mac bindings. > northd: Avoid triggering unnecesary recompute for nb_global changes. > northd: Track lsp names referenced by LB health-check. > northd: Incremental processing of VIF changes in 'northd' node. > northd: Refactor the logic related to "unknown" mc_group. > northd: Incremental processing of VIF additions in 'lflow' node. > northd: Omit alerts for SB lflow and mc-group table. > northd: Split a new node "sync-from-sb" from the "northd" node. > northd: Avoid another round-trip for clearing > additional-chassis-activated. > northd: Incremental processing of SB port_binding in "northd" node. > northd: Ignore "up" column change if ignore_lsp_down is true. > > lib/inc-proc-eng.h | 21 +- > lib/ovn-util.c | 15 + > lib/ovn-util.h | 1 + > northd/automake.mk | 2 + > northd/en-lflow.c | 97 +++- > northd/en-lflow.h | 1 + > northd/en-northd.c | 186 +++++-- > northd/en-northd.h | 4 + > northd/en-sync-from-sb.c | 86 +++ > northd/en-sync-from-sb.h | 11 + > northd/inc-proc-northd.c | 30 +- > northd/northd.c | 1124 ++++++++++++++++++++++++++++++-------- > northd/northd.h | 64 ++- > northd/ovn-northd.c | 8 + > tests/ovn-northd.at | 65 ++- > tests/ovn.at | 26 +- > 16 files changed, 1400 insertions(+), 341 deletions(-) > create mode 100644 northd/en-sync-from-sb.c > create mode 100644 northd/en-sync-from-sb.h > > -- > 2.30.2 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > The whole series looks good to me, thanks! Reviewed-by: Ales Musil <amusil@redhat.com>
This series implements incremental processing for VIF (regular logical switch ports of VM/pod) changes in ovn-northd. The performance gains of a single logical switch port creation has improved significantly. Below are the performance test results simulating an ovn-k8s topology of 500 nodes x 50 lsp per node, with Intel i9-7920X CPU @ 2.90GHz: Before: lsp-del: 955ms lsp-add: 919ms After: lsp-del: 776ms (~20% reduction) lsp-add: 30ms (~97% reduction) Note: the test uses single thread of northd. In addition, the last 5 patches focuses on reducing the number of recomputes end-to-end for the VIF related changes when NB/SB update notifications come back to northd. Before these patches, there were 5 recompute triggered because of the back-and-forth NB/SB updates, and after these changes there are zero recomputes for the VIF creation + binding scenario. Some TODOs: - For VIF deletion/update, the lflow node still recomputes. This will be taken care after implementing hash index on the lflows (needs to be thread-safe and efficient). This is why the lsp-del improvement is only 20%, and also the reason why there is still one recompute for deletion/update. - When there are continuous changes to NB/SB, there can be updates received by ovn-northd when it still has outstanding transactions, and it will fallback to recompute in this situation. This problem needs to be resolved because large scale environment usually has high churn rates. - Relax the constraint for LSP changes to allow for more incremental processing. This can be gradually added according to the real needs. Han Zhou (14): northd: Remove bfd_connections from northd_data. northd: Enable persistence of logical flows between engine runs. inc-proc-eng.h: Refactor I-P engine node macros. ovn.at: Fix test case: delete mac bindings. northd: Avoid triggering unnecesary recompute for nb_global changes. northd: Track lsp names referenced by LB health-check. northd: Incremental processing of VIF changes in 'northd' node. northd: Refactor the logic related to "unknown" mc_group. northd: Incremental processing of VIF additions in 'lflow' node. northd: Omit alerts for SB lflow and mc-group table. northd: Split a new node "sync-from-sb" from the "northd" node. northd: Avoid another round-trip for clearing additional-chassis-activated. northd: Incremental processing of SB port_binding in "northd" node. northd: Ignore "up" column change if ignore_lsp_down is true. lib/inc-proc-eng.h | 21 +- lib/ovn-util.c | 15 + lib/ovn-util.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 97 +++- northd/en-lflow.h | 1 + northd/en-northd.c | 186 +++++-- northd/en-northd.h | 4 + northd/en-sync-from-sb.c | 86 +++ northd/en-sync-from-sb.h | 11 + northd/inc-proc-northd.c | 30 +- northd/northd.c | 1124 ++++++++++++++++++++++++++++++-------- northd/northd.h | 64 ++- northd/ovn-northd.c | 8 + tests/ovn-northd.at | 65 ++- tests/ovn.at | 26 +- 16 files changed, 1400 insertions(+), 341 deletions(-) create mode 100644 northd/en-sync-from-sb.c create mode 100644 northd/en-sync-from-sb.h