Message ID | cover.1562858727.git.lorenzo.bianconi@redhat.com |
---|---|
Headers | show |
Series | OVN: add Controller Events | expand |
On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > There are situations where arrival of certain types of traffic into OVS > does not warrant a "typical" action, such as output to a specific port > or dropping. Rather, the decision about what to do needs to be left to a > CMS. > > The series here introduces a new table, Controller_Event, for this > purpose. Traffic into OVS can raise a 'controller' event that results in > a Controller_Event being written to the southbound database. The > intention is for a CMS to see the events and take some sort of action. > When the CMS has seen the event and taken appropriate action, then it > can remove the correponding row in Controller_Event table. The series seems OK to me. However: do we need to do something to rate-limit sending messages to ovn-controller? It seems like a flood of packets could trigger a flood of OpenFlow messages to ovn-controller, which could result in poor service. I suspect the answer is "yes", but maybe it is "yes, but it's OK for 2.12 and we can add rate-limiting in the next release". What do you think? Thanks, Ben.
> On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > > There are situations where arrival of certain types of traffic into OVS > > does not warrant a "typical" action, such as output to a specific port > > or dropping. Rather, the decision about what to do needs to be left to a > > CMS. > > > > The series here introduces a new table, Controller_Event, for this > > purpose. Traffic into OVS can raise a 'controller' event that results in > > a Controller_Event being written to the southbound database. The > > intention is for a CMS to see the events and take some sort of action. > > When the CMS has seen the event and taken appropriate action, then it > > can remove the correponding row in Controller_Event table. > > The series seems OK to me. However: do we need to do something to > rate-limit sending messages to ovn-controller? It seems like a flood of > packets could trigger a flood of OpenFlow messages to ovn-controller, > which could result in poor service. > > I suspect the answer is "yes", but maybe it is "yes, but it's OK for > 2.12 and we can add rate-limiting in the next release". > > What do you think? Hi Ben, in pinctrl_handle_empty_lb_backends_opts() I added a limit to event_table size. If we overcame this limit, ovn-controller discards the received event and it does not notify main thread. Do you think we need some more sophisticated? If so, and if you agree, I think we can work on it after 2.12 release. Regards, Lorenzo > > Thanks, > > Ben.
On Fri, Jul 12, 2019 at 11:46 AM Lorenzo Bianconi < lorenzo.bianconi@redhat.com> wrote: > > > On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > > > There are situations where arrival of certain types of traffic into OVS > > > does not warrant a "typical" action, such as output to a specific port > > > or dropping. Rather, the decision about what to do needs to be left to a > > > CMS. > > > > > > The series here introduces a new table, Controller_Event, for this > > > purpose. Traffic into OVS can raise a 'controller' event that results in > > > a Controller_Event being written to the southbound database. The > > > intention is for a CMS to see the events and take some sort of action. > > > When the CMS has seen the event and taken appropriate action, then it > > > can remove the correponding row in Controller_Event table. > > > > The series seems OK to me. However: do we need to do something to > > rate-limit sending messages to ovn-controller? It seems like a flood of > > packets could trigger a flood of OpenFlow messages to ovn-controller, > > which could result in poor service. > > > > I suspect the answer is "yes", but maybe it is "yes, but it's OK for > > 2.12 and we can add rate-limiting in the next release". > > > > What do you think? > > Hi Ben, > > in pinctrl_handle_empty_lb_backends_opts() I added a limit to event_table > size. If we overcame this limit, ovn-controller discards the received event > and it does not notify main thread. Do you think we need some more sophisticated? > If so, and if you agree, I think we can work on it after 2.12 release. > > Regards, > Lorenzo > > > > > Thanks, > > > > Ben. > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev Hi Lorenzo, I am worried about the flooding, too. Thanks for adding the limit for event_table. It is good that it won't notify the main thread and update SB DB continuously. However, pinctrl thread would still get overloaded if controller messages keep flooding to ovn-controller. Would it be better to use meter action in the flow to ratelimit the messages to ovn-controller, something like what ACL log ratelimit does? I think it is ok for the ratelimit improvement to be in next release, since the feature is not enabled by default (i.e. controller_event opt in NB global is not set). Thanks, Han
> On Fri, Jul 12, 2019 at 11:46 AM Lorenzo Bianconi < > lorenzo.bianconi@redhat.com> wrote: > > > > > On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > > > > There are situations where arrival of certain types of traffic into > OVS > > > > does not warrant a "typical" action, such as output to a specific port > > > > or dropping. Rather, the decision about what to do needs to be left > to a > > > > CMS. > > > > > > > > The series here introduces a new table, Controller_Event, for this > > > > purpose. Traffic into OVS can raise a 'controller' event that results > in > > > > a Controller_Event being written to the southbound database. The > > > > intention is for a CMS to see the events and take some sort of action. > > > > When the CMS has seen the event and taken appropriate action, then it > > > > can remove the correponding row in Controller_Event table. > > > > > > The series seems OK to me. However: do we need to do something to > > > rate-limit sending messages to ovn-controller? It seems like a flood of > > > packets could trigger a flood of OpenFlow messages to ovn-controller, > > > which could result in poor service. > > > > > > I suspect the answer is "yes", but maybe it is "yes, but it's OK for > > > 2.12 and we can add rate-limiting in the next release". > > > > > > What do you think? > > > > Hi Ben, > > > > in pinctrl_handle_empty_lb_backends_opts() I added a limit to event_table > > size. If we overcame this limit, ovn-controller discards the received > event > > and it does not notify main thread. Do you think we need some more > sophisticated? > > If so, and if you agree, I think we can work on it after 2.12 release. > > > > Regards, > > Lorenzo > > > > > > > > Thanks, > > > > > > Ben. > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > Hi Lorenzo, > > I am worried about the flooding, too. Thanks for adding the limit for > event_table. It is good that it won't notify the main thread and update SB > DB continuously. However, pinctrl thread would still get overloaded if > controller messages keep flooding to ovn-controller. Would it be better to > use meter action in the flow to ratelimit the messages to ovn-controller, > something like what ACL log ratelimit does? > > I think it is ok for the ratelimit improvement to be in next release, since > the feature is not enabled by default (i.e. controller_event opt in NB > global is not set). Thx for the review Han, I will add the ratelimit improvement to my ToDo list :) Regards, Lorenzo > > Thanks, > Han
On Fri, Jul 12, 2019 at 12:36:39PM -0700, Han Zhou wrote: > On Fri, Jul 12, 2019 at 11:46 AM Lorenzo Bianconi < > lorenzo.bianconi@redhat.com> wrote: > > > > > On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > > > > There are situations where arrival of certain types of traffic into > OVS > > > > does not warrant a "typical" action, such as output to a specific port > > > > or dropping. Rather, the decision about what to do needs to be left > to a > > > > CMS. > > > > > > > > The series here introduces a new table, Controller_Event, for this > > > > purpose. Traffic into OVS can raise a 'controller' event that results > in > > > > a Controller_Event being written to the southbound database. The > > > > intention is for a CMS to see the events and take some sort of action. > > > > When the CMS has seen the event and taken appropriate action, then it > > > > can remove the correponding row in Controller_Event table. > > > > > > The series seems OK to me. However: do we need to do something to > > > rate-limit sending messages to ovn-controller? It seems like a flood of > > > packets could trigger a flood of OpenFlow messages to ovn-controller, > > > which could result in poor service. > > > > > > I suspect the answer is "yes", but maybe it is "yes, but it's OK for > > > 2.12 and we can add rate-limiting in the next release". > > > > > > What do you think? > > > > Hi Ben, > > > > in pinctrl_handle_empty_lb_backends_opts() I added a limit to event_table > > size. If we overcame this limit, ovn-controller discards the received > event > > and it does not notify main thread. Do you think we need some more > sophisticated? > > If so, and if you agree, I think we can work on it after 2.12 release. > > > > Regards, > > Lorenzo > > > > > > > > Thanks, > > > > > > Ben. > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > Hi Lorenzo, > > I am worried about the flooding, too. Thanks for adding the limit for > event_table. It is good that it won't notify the main thread and update SB > DB continuously. However, pinctrl thread would still get overloaded if > controller messages keep flooding to ovn-controller. Would it be better to > use meter action in the flow to ratelimit the messages to ovn-controller, > something like what ACL log ratelimit does? That's the kind of overloading I was concerned about too. > I think it is ok for the ratelimit improvement to be in next release, since > the feature is not enabled by default (i.e. controller_event opt in NB > global is not set). Good point. Let's consider this a to-do item then.
On Fri, Jul 12, 2019 at 09:50:04PM +0200, Lorenzo Bianconi wrote: > > On Fri, Jul 12, 2019 at 11:46 AM Lorenzo Bianconi < > > lorenzo.bianconi@redhat.com> wrote: > > > > > > > On Thu, Jul 11, 2019 at 05:48:42PM +0200, Lorenzo Bianconi wrote: > > > > > There are situations where arrival of certain types of traffic into > > OVS > > > > > does not warrant a "typical" action, such as output to a specific port > > > > > or dropping. Rather, the decision about what to do needs to be left > > to a > > > > > CMS. > > > > > > > > > > The series here introduces a new table, Controller_Event, for this > > > > > purpose. Traffic into OVS can raise a 'controller' event that results > > in > > > > > a Controller_Event being written to the southbound database. The > > > > > intention is for a CMS to see the events and take some sort of action. > > > > > When the CMS has seen the event and taken appropriate action, then it > > > > > can remove the correponding row in Controller_Event table. > > > > > > > > The series seems OK to me. However: do we need to do something to > > > > rate-limit sending messages to ovn-controller? It seems like a flood of > > > > packets could trigger a flood of OpenFlow messages to ovn-controller, > > > > which could result in poor service. > > > > > > > > I suspect the answer is "yes", but maybe it is "yes, but it's OK for > > > > 2.12 and we can add rate-limiting in the next release". > > > > > > > > What do you think? > > > > > > Hi Ben, > > > > > > in pinctrl_handle_empty_lb_backends_opts() I added a limit to event_table > > > size. If we overcame this limit, ovn-controller discards the received > > event > > > and it does not notify main thread. Do you think we need some more > > sophisticated? > > > If so, and if you agree, I think we can work on it after 2.12 release. > > > > > > Regards, > > > Lorenzo > > > > > > > > > > > Thanks, > > > > > > > > Ben. > > > _______________________________________________ > > > dev mailing list > > > dev@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > > > Hi Lorenzo, > > > > I am worried about the flooding, too. Thanks for adding the limit for > > event_table. It is good that it won't notify the main thread and update SB > > DB continuously. However, pinctrl thread would still get overloaded if > > controller messages keep flooding to ovn-controller. Would it be better to > > use meter action in the flow to ratelimit the messages to ovn-controller, > > something like what ACL log ratelimit does? > > > > I think it is ok for the ratelimit improvement to be in next release, since > > the feature is not enabled by default (i.e. controller_event opt in NB > > global is not set). > > Thx for the review Han, I will add the ratelimit improvement to my ToDo list :) I applied this series to master.