| Message ID | 1482335131-129907-1-git-send-email-bhanuprakash.bodireddy@intel.com |
|---|---|
| State | Changes Requested |
| Delegated to: | Darrell Ball |
| Headers | show |
Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> writes: > Increase the DPDK pmd thread scheduling priority by lowering the nice > value. This will advise the kernel scheduler to prioritize pmd thread > over other processes. > > Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> > --- Sorry for jumping into this so late. Is there a measured benefit to this patch? Do you have a test case to reproduce the effect you're seeing? Might it be better to write up documentation for the user describing chrt/nice/renice utilities? > v7->v8: > * Rebase > * Update the documentation file @Documentation/intro/install/dpdk-advanced.rst > > v6->v7: > * Remove realtime scheduling policy logic. > * Increase pmd thread scheduling priority by lowering nice value to -20. > * Update doc accordingly. > > v5->v6: > * Prohibit spawning pmd thread on the lowest core in dpdk-lcore-mask if > lcore-mask and pmd-mask affinity are identical. > * Updated Note section in INSTALL.DPDK-ADVANCED doc. > * Tested below cases to verify system stability with pmd priority patch > > v4->v5: > * Reword Note section in DPDK-ADVANCED.md > > v3->v4: > * Document update > * Use ovs_strerror for reporting errors in lib-numa.c > > v2->v3: > * Move set_priority() function to lib/ovs-numa.c > * Apply realtime scheduling policy and priority to pmd thread only if > pmd-cpu-mask is passed. > * Update INSTALL.DPDK-ADVANCED. > > v1->v2: > * Removed #ifdef and introduced dummy function "pmd_thread_setpriority" > in netdev-dpdk.h > * Rebase > > Documentation/intro/install/dpdk-advanced.rst | 8 +++++++- > lib/dpif-netdev.c | 4 ++++ > lib/ovs-numa.c | 19 +++++++++++++++++++ > lib/ovs-numa.h | 1 + > 4 files changed, 31 insertions(+), 1 deletion(-) > > diff --git a/Documentation/intro/install/dpdk-advanced.rst b/Documentation/intro/install/dpdk-advanced.rst > index 44d1cd7..67815ac 100644 > --- a/Documentation/intro/install/dpdk-advanced.rst > +++ b/Documentation/intro/install/dpdk-advanced.rst > @@ -238,7 +238,8 @@ affinitized accordingly. > to be affinitized to isolated cores for optimum performance. > > By setting a bit in the mask, a pmd thread is created and pinned to the > - corresponding CPU core. e.g. to run a pmd thread on core 2:: > + corresponding CPU core and the nice value set to '-20'. > + e.g. to run a pmd thread on core 2:: > > $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 > > @@ -278,6 +279,11 @@ improvements as there will be more total CPU occupancy available:: > > NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 > > + .. note:: > + It is recommended that the OVS control thread and pmd thread shouldn't be > + pinned to the same core i.e 'dpdk-lcore-mask' and 'pmd-cpu-mask' cpu mask > + settings should be non-overlapping. > + > DPDK Physical Port Rx Queues > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c > index 0b73056..f58a855 100644 > --- a/lib/dpif-netdev.c > +++ b/lib/dpif-netdev.c > @@ -3120,6 +3120,10 @@ pmd_thread_main(void *f_) > ovs_numa_thread_setaffinity_core(pmd->core_id); > dpdk_set_lcore_id(pmd->core_id); > poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); > + > + /* Set pmd thread's nice value to -20 */ > +#define MIN_NICE -20 > + ovs_numa_thread_setpriority(MIN_NICE); > reload: > emc_cache_init(&pmd->flow_cache); > > diff --git a/lib/ovs-numa.c b/lib/ovs-numa.c > index e1e7068..d52c517 100644 > --- a/lib/ovs-numa.c > +++ b/lib/ovs-numa.c > @@ -23,6 +23,7 @@ > #include <dirent.h> > #include <stddef.h> > #include <string.h> > +#include <sys/resource.h> > #include <sys/types.h> > #include <unistd.h> > #endif /* __linux__ */ > @@ -620,3 +621,21 @@ int ovs_numa_thread_setaffinity_core(unsigned core_id OVS_UNUSED) > return EOPNOTSUPP; > #endif /* __linux__ */ > } > + > +void > +ovs_numa_thread_setpriority(int nice OVS_UNUSED) > +{ > + if (dummy_numa) { > + return; > + } > + > +#ifndef _WIN32 > + int err; > + err = setpriority(PRIO_PROCESS, 0, nice); > + if (err) { > + VLOG_ERR("Thread priority error %s",ovs_strerror(err)); > + } > +#else > + return EOPNOTSUPP; > +#endif > +} > diff --git a/lib/ovs-numa.h b/lib/ovs-numa.h > index be836b2..3ce13c7 100644 > --- a/lib/ovs-numa.h > +++ b/lib/ovs-numa.h > @@ -56,6 +56,7 @@ void ovs_numa_unpin_core(unsigned core_id); > struct ovs_numa_dump *ovs_numa_dump_cores_on_numa(int numa_id); > void ovs_numa_dump_destroy(struct ovs_numa_dump *); > int ovs_numa_thread_setaffinity_core(unsigned core_id); > +void ovs_numa_thread_setpriority(int nice); > > #define FOR_EACH_CORE_ON_NUMA(ITER, DUMP) \ > LIST_FOR_EACH((ITER), list_node, &(DUMP)->dump)
Hi Bhanu, This patch no longer applies cleanly. $git apply ... error: Documentation/intro/install/dpdk-advanced.rst: No such file or directory error: patch failed: lib/ovs-numa.h:56 error: lib/ovs-numa.h: patch does not apply Also some more information on the rationale behind such a change would be useful. E.g. is it helpful in the case where the PMDs have not been given isolated cores? Cheers, Billy. > -----Original Message----- > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- > bounces@openvswitch.org] On Behalf Of Aaron Conole > Sent: Tuesday, January 3, 2017 8:08 PM > To: Bodireddy, Bhanuprakash <bhanuprakash.bodireddy@intel.com> > Cc: dev@openvswitch.org > Subject: Re: [ovs-dev] [PATCH v8] netdev-dpdk: Increase pmd thread priority > > Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> writes: > > > Increase the DPDK pmd thread scheduling priority by lowering the nice > > value. This will advise the kernel scheduler to prioritize pmd thread > > over other processes. > > > > Signed-off-by: Bhanuprakash Bodireddy > > <bhanuprakash.bodireddy@intel.com> > > --- > > Sorry for jumping into this so late. Is there a measured benefit to this patch? > Do you have a test case to reproduce the effect you're seeing? Might it be > better to write up documentation for the user describing chrt/nice/renice > utilities? > > > v7->v8: > > * Rebase > > * Update the documentation file > > @Documentation/intro/install/dpdk-advanced.rst > > > > v6->v7: > > * Remove realtime scheduling policy logic. > > * Increase pmd thread scheduling priority by lowering nice value to -20. > > * Update doc accordingly. > > > > v5->v6: > > * Prohibit spawning pmd thread on the lowest core in dpdk-lcore-mask if > > lcore-mask and pmd-mask affinity are identical. > > * Updated Note section in INSTALL.DPDK-ADVANCED doc. > > * Tested below cases to verify system stability with pmd priority > > patch > > > > v4->v5: > > * Reword Note section in DPDK-ADVANCED.md > > > > v3->v4: > > * Document update > > * Use ovs_strerror for reporting errors in lib-numa.c > > > > v2->v3: > > * Move set_priority() function to lib/ovs-numa.c > > * Apply realtime scheduling policy and priority to pmd thread only if > > pmd-cpu-mask is passed. > > * Update INSTALL.DPDK-ADVANCED. > > > > v1->v2: > > * Removed #ifdef and introduced dummy function > "pmd_thread_setpriority" > > in netdev-dpdk.h > > * Rebase > > > > Documentation/intro/install/dpdk-advanced.rst | 8 +++++++- > > lib/dpif-netdev.c | 4 ++++ > > lib/ovs-numa.c | 19 +++++++++++++++++++ > > lib/ovs-numa.h | 1 + > > 4 files changed, 31 insertions(+), 1 deletion(-) > > > > diff --git a/Documentation/intro/install/dpdk-advanced.rst > > b/Documentation/intro/install/dpdk-advanced.rst > > index 44d1cd7..67815ac 100644 > > --- a/Documentation/intro/install/dpdk-advanced.rst > > +++ b/Documentation/intro/install/dpdk-advanced.rst > > @@ -238,7 +238,8 @@ affinitized accordingly. > > to be affinitized to isolated cores for optimum performance. > > > > By setting a bit in the mask, a pmd thread is created and pinned to > > the > > - corresponding CPU core. e.g. to run a pmd thread on core 2:: > > + corresponding CPU core and the nice value set to '-20'. > > + e.g. to run a pmd thread on core 2:: > > > > $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 > > > > @@ -278,6 +279,11 @@ improvements as there will be more total CPU > occupancy available:: > > > > NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 > > > > + .. note:: > > + It is recommended that the OVS control thread and pmd thread > shouldn't be > > + pinned to the same core i.e 'dpdk-lcore-mask' and 'pmd-cpu-mask' cpu > mask > > + settings should be non-overlapping. > > + > > DPDK Physical Port Rx Queues > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index > > 0b73056..f58a855 100644 > > --- a/lib/dpif-netdev.c > > +++ b/lib/dpif-netdev.c > > @@ -3120,6 +3120,10 @@ pmd_thread_main(void *f_) > > ovs_numa_thread_setaffinity_core(pmd->core_id); > > dpdk_set_lcore_id(pmd->core_id); > > poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); > > + > > + /* Set pmd thread's nice value to -20 */ #define MIN_NICE -20 > > + ovs_numa_thread_setpriority(MIN_NICE); > > reload: > > emc_cache_init(&pmd->flow_cache); > > > > diff --git a/lib/ovs-numa.c b/lib/ovs-numa.c index e1e7068..d52c517 > > 100644 > > --- a/lib/ovs-numa.c > > +++ b/lib/ovs-numa.c > > @@ -23,6 +23,7 @@ > > #include <dirent.h> > > #include <stddef.h> > > #include <string.h> > > +#include <sys/resource.h> > > #include <sys/types.h> > > #include <unistd.h> > > #endif /* __linux__ */ > > @@ -620,3 +621,21 @@ int ovs_numa_thread_setaffinity_core(unsigned > core_id OVS_UNUSED) > > return EOPNOTSUPP; > > #endif /* __linux__ */ > > } > > + > > +void > > +ovs_numa_thread_setpriority(int nice OVS_UNUSED) { > > + if (dummy_numa) { > > + return; > > + } > > + > > +#ifndef _WIN32 > > + int err; > > + err = setpriority(PRIO_PROCESS, 0, nice); > > + if (err) { > > + VLOG_ERR("Thread priority error %s",ovs_strerror(err)); > > + } > > +#else > > + return EOPNOTSUPP; > > +#endif > > +} > > diff --git a/lib/ovs-numa.h b/lib/ovs-numa.h index be836b2..3ce13c7 > > 100644 > > --- a/lib/ovs-numa.h > > +++ b/lib/ovs-numa.h > > @@ -56,6 +56,7 @@ void ovs_numa_unpin_core(unsigned core_id); struct > > ovs_numa_dump *ovs_numa_dump_cores_on_numa(int numa_id); void > > ovs_numa_dump_destroy(struct ovs_numa_dump *); int > > ovs_numa_thread_setaffinity_core(unsigned core_id); > > +void ovs_numa_thread_setpriority(int nice); > > > > #define FOR_EACH_CORE_ON_NUMA(ITER, DUMP) \ > > LIST_FOR_EACH((ITER), list_node, &(DUMP)->dump) > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
On Tue, Jan 03, 2017 at 03:08:11PM -0500, Aaron Conole wrote: > Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> writes: > > > Increase the DPDK pmd thread scheduling priority by lowering the nice > > value. This will advise the kernel scheduler to prioritize pmd thread > > over other processes. > > > > Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> > > --- > > Sorry for jumping into this so late. Is there a measured benefit to > this patch? Do you have a test case to reproduce the effect you're > seeing? Might it be better to write up documentation for the user > describing chrt/nice/renice utilities? I haven't tested this myself but I presume that on a system without any task or CPU isolation tuning, this would help the PMD to provide better performance out-of-the-box. -- Flavio
Hi Billy, >Hi Bhanu, > >This patch no longer applies cleanly. Thanks for looking in to this patch. It's pretty old patch and should be rebased. > >$git apply ... >error: Documentation/intro/install/dpdk-advanced.rst: No such file or >directory >error: patch failed: lib/ovs-numa.h:56 >error: lib/ovs-numa.h: patch does not apply > >Also some more information on the rationale behind such a change would be >useful. E.g. is it helpful in the case where the PMDs have not been given >isolated cores? As you would have already noticed, this patch was at v8. Initially I started out with applying real time scheduling policy and priority to PMD threads. It was noticed that real time priorities(the original implementation) could potentially cause problems in few corner cases. Flavio and Daniele suggested to bump up the PMD thread priority instead of applying real time scheduling policy. This is what was implemented in the last sent v8. With datapath threads having higher priority the overall switching performance wouldn't suffer and may improve in out-of-box deployments as mentioned by Flavio in other mail. I will rebase and send the patch. - Bhanuprakash. > >> -----Original Message----- >> From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- >> bounces@openvswitch.org] On Behalf Of Aaron Conole >> Sent: Tuesday, January 3, 2017 8:08 PM >> To: Bodireddy, Bhanuprakash <bhanuprakash.bodireddy@intel.com> >> Cc: dev@openvswitch.org >> Subject: Re: [ovs-dev] [PATCH v8] netdev-dpdk: Increase pmd thread >> priority >> >> Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> writes: >> >> > Increase the DPDK pmd thread scheduling priority by lowering the >> > nice value. This will advise the kernel scheduler to prioritize pmd >> > thread over other processes. >> > >> > Signed-off-by: Bhanuprakash Bodireddy >> > <bhanuprakash.bodireddy@intel.com> >> > --- >> >> Sorry for jumping into this so late. Is there a measured benefit to this >patch? >> Do you have a test case to reproduce the effect you're seeing? Might >> it be better to write up documentation for the user describing >> chrt/nice/renice utilities? >> >> > v7->v8: >> > * Rebase >> > * Update the documentation file >> > @Documentation/intro/install/dpdk-advanced.rst >> > >> > v6->v7: >> > * Remove realtime scheduling policy logic. >> > * Increase pmd thread scheduling priority by lowering nice value to -20. >> > * Update doc accordingly. >> > >> > v5->v6: >> > * Prohibit spawning pmd thread on the lowest core in dpdk-lcore-mask if >> > lcore-mask and pmd-mask affinity are identical. >> > * Updated Note section in INSTALL.DPDK-ADVANCED doc. >> > * Tested below cases to verify system stability with pmd priority >> > patch >> > >> > v4->v5: >> > * Reword Note section in DPDK-ADVANCED.md >> > >> > v3->v4: >> > * Document update >> > * Use ovs_strerror for reporting errors in lib-numa.c >> > >> > v2->v3: >> > * Move set_priority() function to lib/ovs-numa.c >> > * Apply realtime scheduling policy and priority to pmd thread only if >> > pmd-cpu-mask is passed. >> > * Update INSTALL.DPDK-ADVANCED. >> > >> > v1->v2: >> > * Removed #ifdef and introduced dummy function >> "pmd_thread_setpriority" >> > in netdev-dpdk.h >> > * Rebase >> > >> > Documentation/intro/install/dpdk-advanced.rst | 8 +++++++- >> > lib/dpif-netdev.c | 4 ++++ >> > lib/ovs-numa.c | 19 +++++++++++++++++++ >> > lib/ovs-numa.h | 1 + >> > 4 files changed, 31 insertions(+), 1 deletion(-) >> > >> > diff --git a/Documentation/intro/install/dpdk-advanced.rst >> > b/Documentation/intro/install/dpdk-advanced.rst >> > index 44d1cd7..67815ac 100644 >> > --- a/Documentation/intro/install/dpdk-advanced.rst >> > +++ b/Documentation/intro/install/dpdk-advanced.rst >> > @@ -238,7 +238,8 @@ affinitized accordingly. >> > to be affinitized to isolated cores for optimum performance. >> > >> > By setting a bit in the mask, a pmd thread is created and pinned >> > to the >> > - corresponding CPU core. e.g. to run a pmd thread on core 2:: >> > + corresponding CPU core and the nice value set to '-20'. >> > + e.g. to run a pmd thread on core 2:: >> > >> > $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 >> > >> > @@ -278,6 +279,11 @@ improvements as there will be more total CPU >> occupancy available:: >> > >> > NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 >> > >> > + .. note:: >> > + It is recommended that the OVS control thread and pmd thread >> shouldn't be >> > + pinned to the same core i.e 'dpdk-lcore-mask' and >> > + 'pmd-cpu-mask' cpu >> mask >> > + settings should be non-overlapping. >> > + >> > DPDK Physical Port Rx Queues >> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > >> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index >> > 0b73056..f58a855 100644 >> > --- a/lib/dpif-netdev.c >> > +++ b/lib/dpif-netdev.c >> > @@ -3120,6 +3120,10 @@ pmd_thread_main(void *f_) >> > ovs_numa_thread_setaffinity_core(pmd->core_id); >> > dpdk_set_lcore_id(pmd->core_id); >> > poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); >> > + >> > + /* Set pmd thread's nice value to -20 */ #define MIN_NICE -20 >> > + ovs_numa_thread_setpriority(MIN_NICE); >> > reload: >> > emc_cache_init(&pmd->flow_cache); >> > >> > diff --git a/lib/ovs-numa.c b/lib/ovs-numa.c index e1e7068..d52c517 >> > 100644 >> > --- a/lib/ovs-numa.c >> > +++ b/lib/ovs-numa.c >> > @@ -23,6 +23,7 @@ >> > #include <dirent.h> >> > #include <stddef.h> >> > #include <string.h> >> > +#include <sys/resource.h> >> > #include <sys/types.h> >> > #include <unistd.h> >> > #endif /* __linux__ */ >> > @@ -620,3 +621,21 @@ int ovs_numa_thread_setaffinity_core(unsigned >> core_id OVS_UNUSED) >> > return EOPNOTSUPP; >> > #endif /* __linux__ */ >> > } >> > + >> > +void >> > +ovs_numa_thread_setpriority(int nice OVS_UNUSED) { >> > + if (dummy_numa) { >> > + return; >> > + } >> > + >> > +#ifndef _WIN32 >> > + int err; >> > + err = setpriority(PRIO_PROCESS, 0, nice); >> > + if (err) { >> > + VLOG_ERR("Thread priority error %s",ovs_strerror(err)); >> > + } >> > +#else >> > + return EOPNOTSUPP; >> > +#endif >> > +} >> > diff --git a/lib/ovs-numa.h b/lib/ovs-numa.h index be836b2..3ce13c7 >> > 100644 >> > --- a/lib/ovs-numa.h >> > +++ b/lib/ovs-numa.h >> > @@ -56,6 +56,7 @@ void ovs_numa_unpin_core(unsigned core_id); >> > struct ovs_numa_dump *ovs_numa_dump_cores_on_numa(int >numa_id); >> > void ovs_numa_dump_destroy(struct ovs_numa_dump *); int >> > ovs_numa_thread_setaffinity_core(unsigned core_id); >> > +void ovs_numa_thread_setpriority(int nice); >> > >> > #define FOR_EACH_CORE_ON_NUMA(ITER, DUMP) \ >> > LIST_FOR_EACH((ITER), list_node, &(DUMP)->dump) >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff --git a/Documentation/intro/install/dpdk-advanced.rst b/Documentation/intro/install/dpdk-advanced.rst index 44d1cd7..67815ac 100644 --- a/Documentation/intro/install/dpdk-advanced.rst +++ b/Documentation/intro/install/dpdk-advanced.rst @@ -238,7 +238,8 @@ affinitized accordingly. to be affinitized to isolated cores for optimum performance. By setting a bit in the mask, a pmd thread is created and pinned to the - corresponding CPU core. e.g. to run a pmd thread on core 2:: + corresponding CPU core and the nice value set to '-20'. + e.g. to run a pmd thread on core 2:: $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 @@ -278,6 +279,11 @@ improvements as there will be more total CPU occupancy available:: NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 + .. note:: + It is recommended that the OVS control thread and pmd thread shouldn't be + pinned to the same core i.e 'dpdk-lcore-mask' and 'pmd-cpu-mask' cpu mask + settings should be non-overlapping. + DPDK Physical Port Rx Queues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 0b73056..f58a855 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -3120,6 +3120,10 @@ pmd_thread_main(void *f_) ovs_numa_thread_setaffinity_core(pmd->core_id); dpdk_set_lcore_id(pmd->core_id); poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); + + /* Set pmd thread's nice value to -20 */ +#define MIN_NICE -20 + ovs_numa_thread_setpriority(MIN_NICE); reload: emc_cache_init(&pmd->flow_cache); diff --git a/lib/ovs-numa.c b/lib/ovs-numa.c index e1e7068..d52c517 100644 --- a/lib/ovs-numa.c +++ b/lib/ovs-numa.c @@ -23,6 +23,7 @@ #include <dirent.h> #include <stddef.h> #include <string.h> +#include <sys/resource.h> #include <sys/types.h> #include <unistd.h> #endif /* __linux__ */ @@ -620,3 +621,21 @@ int ovs_numa_thread_setaffinity_core(unsigned core_id OVS_UNUSED) return EOPNOTSUPP; #endif /* __linux__ */ } + +void +ovs_numa_thread_setpriority(int nice OVS_UNUSED) +{ + if (dummy_numa) { + return; + } + +#ifndef _WIN32 + int err; + err = setpriority(PRIO_PROCESS, 0, nice); + if (err) { + VLOG_ERR("Thread priority error %s",ovs_strerror(err)); + } +#else + return EOPNOTSUPP; +#endif +} diff --git a/lib/ovs-numa.h b/lib/ovs-numa.h index be836b2..3ce13c7 100644 --- a/lib/ovs-numa.h +++ b/lib/ovs-numa.h @@ -56,6 +56,7 @@ void ovs_numa_unpin_core(unsigned core_id); struct ovs_numa_dump *ovs_numa_dump_cores_on_numa(int numa_id); void ovs_numa_dump_destroy(struct ovs_numa_dump *); int ovs_numa_thread_setaffinity_core(unsigned core_id); +void ovs_numa_thread_setpriority(int nice); #define FOR_EACH_CORE_ON_NUMA(ITER, DUMP) \ LIST_FOR_EACH((ITER), list_node, &(DUMP)->dump)
Increase the DPDK pmd thread scheduling priority by lowering the nice value. This will advise the kernel scheduler to prioritize pmd thread over other processes. Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com> --- v7->v8: * Rebase * Update the documentation file @Documentation/intro/install/dpdk-advanced.rst v6->v7: * Remove realtime scheduling policy logic. * Increase pmd thread scheduling priority by lowering nice value to -20. * Update doc accordingly. v5->v6: * Prohibit spawning pmd thread on the lowest core in dpdk-lcore-mask if lcore-mask and pmd-mask affinity are identical. * Updated Note section in INSTALL.DPDK-ADVANCED doc. * Tested below cases to verify system stability with pmd priority patch v4->v5: * Reword Note section in DPDK-ADVANCED.md v3->v4: * Document update * Use ovs_strerror for reporting errors in lib-numa.c v2->v3: * Move set_priority() function to lib/ovs-numa.c * Apply realtime scheduling policy and priority to pmd thread only if pmd-cpu-mask is passed. * Update INSTALL.DPDK-ADVANCED. v1->v2: * Removed #ifdef and introduced dummy function "pmd_thread_setpriority" in netdev-dpdk.h * Rebase Documentation/intro/install/dpdk-advanced.rst | 8 +++++++- lib/dpif-netdev.c | 4 ++++ lib/ovs-numa.c | 19 +++++++++++++++++++ lib/ovs-numa.h | 1 + 4 files changed, 31 insertions(+), 1 deletion(-)