| Message ID | 20190228161357.2044-1-fbl@sysclose.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev] rhel: limit stack size to 2M. | expand |
On Thu, 28 Feb 2019 13:13:57 -0300 Flavio Leitner <fbl@sysclose.org> wrote: > The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd > daemon starts and uses --mlockall (default), it will dirty all memory > regions for all threads which is proportionally to the number of CPUs. > > On a big host this increases memory usage to many hundreds of megabytes > while OVS actually requires much less. > > This patch relies on systemd to limit to 2M/thread. That is much more > than the minimum documented at function ovs_thread_create(): > > /* Some small systems use a default stack size as small as 80 kB, but OVS > * requires approximately 384 kB according to the following analysis: > * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html > * > * We use 512 kB to give us some margin of error. */ > > Signed-off-by: Flavio Leitner <fbl@sysclose.org> > --- > rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 + > 1 file changed, 1 insertion(+) Without the patch: # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits Max stack size 8388608 unlimited bytes # With the patch: # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits Max stack size 2097152 2097152 bytes # Acked-By: Timothy Redaelli <tredaelli@redhat.com> Tested-By: Timothy Redaelli <tredaelli@redhat.com>
On Thu, Feb 28, 2019 at 07:28:58PM +0100, Timothy Redaelli wrote: > On Thu, 28 Feb 2019 13:13:57 -0300 > Flavio Leitner <fbl@sysclose.org> wrote: > > > The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd > > daemon starts and uses --mlockall (default), it will dirty all memory > > regions for all threads which is proportionally to the number of CPUs. > > > > On a big host this increases memory usage to many hundreds of megabytes > > while OVS actually requires much less. > > > > This patch relies on systemd to limit to 2M/thread. That is much more > > than the minimum documented at function ovs_thread_create(): > > > > /* Some small systems use a default stack size as small as 80 kB, but OVS > > * requires approximately 384 kB according to the following analysis: > > * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html > > * > > * We use 512 kB to give us some margin of error. */ > > > > Signed-off-by: Flavio Leitner <fbl@sysclose.org> > > --- > > rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 + > > 1 file changed, 1 insertion(+) > > Without the patch: > > # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits > Max stack size 8388608 unlimited bytes > # > > With the patch: > > # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits > Max stack size 2097152 2097152 bytes > # > > Acked-By: Timothy Redaelli <tredaelli@redhat.com> > Tested-By: Timothy Redaelli <tredaelli@redhat.com> Applied to master, thanks Flavio (and Timothy)!
diff --git a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in index 525deae0b..317aa993c 100644 --- a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in +++ b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in @@ -14,6 +14,7 @@ Environment=XDG_RUNTIME_DIR=/var/run/openvswitch EnvironmentFile=/etc/openvswitch/default.conf EnvironmentFile=-/etc/sysconfig/openvswitch EnvironmentFile=-/run/openvswitch/useropts +LimitSTACK=2M @begin_dpdk@ ExecStartPre=-/bin/sh -c '/usr/bin/chown :$${OVS_USER_ID##*:} /dev/hugepages' ExecStartPre=-/usr/bin/chmod 0775 /dev/hugepages
The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd daemon starts and uses --mlockall (default), it will dirty all memory regions for all threads which is proportionally to the number of CPUs. On a big host this increases memory usage to many hundreds of megabytes while OVS actually requires much less. This patch relies on systemd to limit to 2M/thread. That is much more than the minimum documented at function ovs_thread_create(): /* Some small systems use a default stack size as small as 80 kB, but OVS * requires approximately 384 kB according to the following analysis: * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html * * We use 512 kB to give us some margin of error. */ Signed-off-by: Flavio Leitner <fbl@sysclose.org> --- rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 + 1 file changed, 1 insertion(+)