| Message ID | 20241122084056.2736306-1-amusil@redhat.com |
|---|---|
| State | New |
| Delegated to: | Ilya Maximets |
| Headers | show |
| Series | [ovs-dev] dpif-netdev: Increase the recirculation limit to 12. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
On 11/22/24 09:40, Ales Musil wrote: > Increase the recirculaton limit to 12 as OVN wants to adopt > strategy which might lead to more recirculations in the logical > router pipeline. Hi, Ales. Could you, please, explain the scenario in more details where so many recirculations are needed? Also, do you expect all the traffic to have increased number of them or is it just for very specific control traffic? Asking because 12 is a lot. Every re-circulation means packet re-classification and potential re-parsing, so it will have a huge performance impact. And it doesn't only ally to the userspace datapath, while kernel might endure so many recirculations in some scenarios, performance will still be awful. Best regards, Ilya Maximets.
On Mon, Dec 2, 2024 at 8:37 PM Ilya Maximets <i.maximets@ovn.org> wrote: > On 11/22/24 09:40, Ales Musil wrote: > > Increase the recirculaton limit to 12 as OVN wants to adopt > > strategy which might lead to more recirculations in the logical > > router pipeline. > > Hi, Ales. Hi Ilya, > Could you, please, explain the scenario in more details > where so many recirculations are needed? Also, do you expect all > the traffic to have increased number of them or is it just for very > specific control traffic? > The need for more recirculations comes from the commit all patch for OVN. Unfortunately almost all traffic will be going through more recirculations, not necessarily 12. > > Asking because 12 is a lot. Every re-circulation means packet > re-classification and potential re-parsing, so it will have a huge > performance impact. > > And it doesn't only ally to the userspace datapath, while kernel > might endure so many recirculations in some scenarios, performance > will still be awful. > I'm aware of the potential performance impact, Han is testing whether we have any performance impact and if it's acceptable. If not we will have to come up with different strategy how to fix a few bugs in OVN. > > Best regards, Ilya Maximets. > > Thanks, Ales
On Tue, 3 Dec 2024 08:13:35 +0100 Ales Musil <amusil@redhat.com> wrote: > On Mon, Dec 2, 2024 at 8:37 PM Ilya Maximets <i.maximets@ovn.org> > wrote: > > > On 11/22/24 09:40, Ales Musil wrote: > > > Increase the recirculaton limit to 12 as OVN wants to adopt > > > strategy which might lead to more recirculations in the logical > > > router pipeline. > > > > Hi, Ales. > > > Hi Ilya, > > > > Could you, please, explain the scenario in more details > > where so many recirculations are needed? Also, do you expect all > > the traffic to have increased number of them or is it just for very > > specific control traffic? > > > > The need for more recirculations comes from the commit > all patch for OVN. Unfortunately almost all traffic will be > going through more recirculations, not necessarily 12. Could you please point me to the patch? > > Asking because 12 is a lot. Every re-circulation means packet > > re-classification and potential re-parsing, so it will have a huge > > performance impact. > > > > And it doesn't only ally to the userspace datapath, while kernel > > might endure so many recirculations in some scenarios, performance > > will still be awful. > > > > I'm aware of the potential performance impact, Han is testing > whether we have any performance impact and if it's acceptable. > If not we will have to come up with different strategy how to fix > a few bugs in OVN. Doesn't that increase the stack usage as well? Thanks, fbl
On Tue, Dec 3, 2024 at 12:30 PM Flavio Leitner <fbl@sysclose.org> wrote: > On Tue, 3 Dec 2024 08:13:35 +0100 > Ales Musil <amusil@redhat.com> wrote: > > > On Mon, Dec 2, 2024 at 8:37 PM Ilya Maximets <i.maximets@ovn.org> > > wrote: > > > > > On 11/22/24 09:40, Ales Musil wrote: > > > > Increase the recirculaton limit to 12 as OVN wants to adopt > > > > strategy which might lead to more recirculations in the logical > > > > router pipeline. > > > > > > Hi, Ales. > > > > > > Hi Ilya, > > > > > > > Could you, please, explain the scenario in more details > > > where so many recirculations are needed? Also, do you expect all > > > the traffic to have increased number of them or is it just for very > > > specific control traffic? > > > > > > > The need for more recirculations comes from the commit > > all patch for OVN. Unfortunately almost all traffic will be > > going through more recirculations, not necessarily 12. > > > Could you please point me to the patch? > It's https://mail.openvswitch.org/pipermail/ovs-dev/2024-November/418506.html. > > > > Asking because 12 is a lot. Every re-circulation means packet > > > re-classification and potential re-parsing, so it will have a huge > > > performance impact. > > > > > > And it doesn't only ally to the userspace datapath, while kernel > > > might endure so many recirculations in some scenarios, performance > > > will still be awful. > > > > > > > I'm aware of the potential performance impact, Han is testing > > whether we have any performance impact and if it's acceptable. > > If not we will have to come up with different strategy how to fix > > a few bugs in OVN. > > > Doesn't that increase the stack usage as well? > > I'm not sure if I know the answer to that. > > Thanks, > fbl > > Thanks, Ales
On Tue, 3 Dec 2024 12:41:53 +0100 Ales Musil <amusil@redhat.com> wrote: > On Tue, Dec 3, 2024 at 12:30 PM Flavio Leitner <fbl@sysclose.org> > wrote: > > > On Tue, 3 Dec 2024 08:13:35 +0100 > > Ales Musil <amusil@redhat.com> wrote: > > > > > On Mon, Dec 2, 2024 at 8:37 PM Ilya Maximets <i.maximets@ovn.org> > > > wrote: > > > > > > > On 11/22/24 09:40, Ales Musil wrote: > > > > > Increase the recirculaton limit to 12 as OVN wants to adopt > > > > > strategy which might lead to more recirculations in the > > > > > logical router pipeline. > > > > > > > > Hi, Ales. > > > > > > > > > Hi Ilya, > > > > > > > > > > Could you, please, explain the scenario in more details > > > > where so many recirculations are needed? Also, do you expect > > > > all the traffic to have increased number of them or is it just > > > > for very specific control traffic? > > > > > > > > > > The need for more recirculations comes from the commit > > > all patch for OVN. Unfortunately almost all traffic will be > > > going through more recirculations, not necessarily 12. > > > > > > Could you please point me to the patch? > > > > > It's > https://mail.openvswitch.org/pipermail/ovs-dev/2024-November/418506.html. > > > > > > > > Asking because 12 is a lot. Every re-circulation means packet > > > > re-classification and potential re-parsing, so it will have a > > > > huge performance impact. > > > > > > > > And it doesn't only ally to the userspace datapath, while kernel > > > > might endure so many recirculations in some scenarios, > > > > performance will still be awful. > > > > > > > > > > I'm aware of the potential performance impact, Han is testing > > > whether we have any performance impact and if it's acceptable. > > > If not we will have to come up with different strategy how to fix > > > a few bugs in OVN. > > > > > > Doesn't that increase the stack usage as well? > > > > > I'm not sure if I know the answer to that. Aaron, maybe you know the answer to that? What was the platform that stack usage was a problem? Thanks, fbl
Flavio Leitner <fbl@sysclose.org> writes: > On Tue, 3 Dec 2024 12:41:53 +0100 > Ales Musil <amusil@redhat.com> wrote: > >> On Tue, Dec 3, 2024 at 12:30 PM Flavio Leitner <fbl@sysclose.org> >> wrote: >> >> > On Tue, 3 Dec 2024 08:13:35 +0100 >> > Ales Musil <amusil@redhat.com> wrote: >> > >> > > On Mon, Dec 2, 2024 at 8:37 PM Ilya Maximets <i.maximets@ovn.org> >> > > wrote: >> > > >> > > > On 11/22/24 09:40, Ales Musil wrote: >> > > > > Increase the recirculaton limit to 12 as OVN wants to adopt >> > > > > strategy which might lead to more recirculations in the >> > > > > logical router pipeline. >> > > > >> > > > Hi, Ales. >> > > >> > > >> > > Hi Ilya, >> > > >> > > >> > > > Could you, please, explain the scenario in more details >> > > > where so many recirculations are needed? Also, do you expect >> > > > all the traffic to have increased number of them or is it just >> > > > for very specific control traffic? >> > > > >> > > >> > > The need for more recirculations comes from the commit >> > > all patch for OVN. Unfortunately almost all traffic will be >> > > going through more recirculations, not necessarily 12. >> > >> > >> > Could you please point me to the patch? >> > >> >> >> It's >> https://mail.openvswitch.org/pipermail/ovs-dev/2024-November/418506.html. >> >> >> > >> > > > Asking because 12 is a lot. Every re-circulation means packet >> > > > re-classification and potential re-parsing, so it will have a >> > > > huge performance impact. >> > > > >> > > > And it doesn't only ally to the userspace datapath, while kernel >> > > > might endure so many recirculations in some scenarios, >> > > > performance will still be awful. >> > > > >> > > >> > > I'm aware of the potential performance impact, Han is testing >> > > whether we have any performance impact and if it's acceptable. >> > > If not we will have to come up with different strategy how to fix >> > > a few bugs in OVN. >> > >> > >> > Doesn't that increase the stack usage as well? >> > >> > >> I'm not sure if I know the answer to that. > > > Aaron, maybe you know the answer to that? Sortof. It can introduce more recursive calls to the clone_execute function, and that will grow stack usage (as well as possibly exceed the number of pre-allocated keys we have for this work and lead to more deferred actions). We'd need to see what the flows look like, but I also am concerned about the amount of additional processing time it would take (since we would need to reparse the key anyway). > What was the platform that stack usage was a problem? There were issues reported in ppc64le systems - but that was because the receive function would be allocated more keys on the stack. Luckily, this doesn't happen anywhere else during the execute path, but it is still concerning. > Thanks, > fbl
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 2a529f272..b899a03bc 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -99,7 +99,7 @@ VLOG_DEFINE_THIS_MODULE(dpif_netdev); #define FLOW_DUMP_MAX_BATCH 50 /* Use per thread recirc_depth to prevent recirculation loop. */ -#define MAX_RECIRC_DEPTH 8 +#define MAX_RECIRC_DEPTH 12 DEFINE_STATIC_PER_THREAD_DATA(uint32_t, recirc_depth, 0) /* Use instant packet send by default. */
Increase the recirculaton limit to 12 as OVN wants to adopt strategy which might lead to more recirculations in the logical router pipeline. Signed-off-by: Ales Musil <amusil@redhat.com> --- lib/dpif-netdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)