[ovs-dev] rhel: limit stack size to 2M.
diff mbox series

Message ID 20190228161357.2044-1-fbl@sysclose.org
State New
Headers show
Series
  • [ovs-dev] rhel: limit stack size to 2M.
Related show

Commit Message

Flavio Leitner Feb. 28, 2019, 4:13 p.m. UTC
The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd
daemon starts and uses --mlockall (default), it will dirty all memory
regions for all threads which is proportionally to the number of CPUs.

On a big host this increases memory usage to many hundreds of megabytes
while OVS actually requires much less.

This patch relies on systemd to limit to 2M/thread. That is much more
than the minimum documented at function ovs_thread_create():

    /* Some small systems use a default stack size as small as 80 kB, but OVS
     * requires approximately 384 kB according to the following analysis:
     * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html
     *
     * We use 512 kB to give us some margin of error. */

Signed-off-by: Flavio Leitner <fbl@sysclose.org>
---
 rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 +
 1 file changed, 1 insertion(+)

Comments

Timothy Redaelli Feb. 28, 2019, 6:28 p.m. UTC | #1
On Thu, 28 Feb 2019 13:13:57 -0300
Flavio Leitner <fbl@sysclose.org> wrote:

> The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd
> daemon starts and uses --mlockall (default), it will dirty all memory
> regions for all threads which is proportionally to the number of CPUs.
> 
> On a big host this increases memory usage to many hundreds of megabytes
> while OVS actually requires much less.
> 
> This patch relies on systemd to limit to 2M/thread. That is much more
> than the minimum documented at function ovs_thread_create():
> 
>     /* Some small systems use a default stack size as small as 80 kB, but OVS
>      * requires approximately 384 kB according to the following analysis:
>      * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html
>      *
>      * We use 512 kB to give us some margin of error. */
> 
> Signed-off-by: Flavio Leitner <fbl@sysclose.org>
> ---
>  rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 +
>  1 file changed, 1 insertion(+)

Without the patch:

# fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits
Max stack size            8388608              unlimited            bytes     
#

With the patch:

# fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits
Max stack size            2097152              2097152              bytes     
#

Acked-By: Timothy Redaelli <tredaelli@redhat.com>
Tested-By: Timothy Redaelli <tredaelli@redhat.com>
Ben Pfaff Feb. 28, 2019, 7:29 p.m. UTC | #2
On Thu, Feb 28, 2019 at 07:28:58PM +0100, Timothy Redaelli wrote:
> On Thu, 28 Feb 2019 13:13:57 -0300
> Flavio Leitner <fbl@sysclose.org> wrote:
> 
> > The default stack size in Fedora/RHEL is 8M, which means when ovs-vswitchd
> > daemon starts and uses --mlockall (default), it will dirty all memory
> > regions for all threads which is proportionally to the number of CPUs.
> > 
> > On a big host this increases memory usage to many hundreds of megabytes
> > while OVS actually requires much less.
> > 
> > This patch relies on systemd to limit to 2M/thread. That is much more
> > than the minimum documented at function ovs_thread_create():
> > 
> >     /* Some small systems use a default stack size as small as 80 kB, but OVS
> >      * requires approximately 384 kB according to the following analysis:
> >      * https://mail.openvswitch.org/pipermail/ovs-dev/2016-January/308592.html
> >      *
> >      * We use 512 kB to give us some margin of error. */
> > 
> > Signed-off-by: Flavio Leitner <fbl@sysclose.org>
> > ---
> >  rhel/usr_lib_systemd_system_ovs-vswitchd.service.in | 1 +
> >  1 file changed, 1 insertion(+)
> 
> Without the patch:
> 
> # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits
> Max stack size            8388608              unlimited            bytes     
> #
> 
> With the patch:
> 
> # fgrep stack /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits
> Max stack size            2097152              2097152              bytes     
> #
> 
> Acked-By: Timothy Redaelli <tredaelli@redhat.com>
> Tested-By: Timothy Redaelli <tredaelli@redhat.com>

Applied to master, thanks Flavio (and Timothy)!

Patch
diff mbox series

diff --git a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
index 525deae0b..317aa993c 100644
--- a/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
+++ b/rhel/usr_lib_systemd_system_ovs-vswitchd.service.in
@@ -14,6 +14,7 @@  Environment=XDG_RUNTIME_DIR=/var/run/openvswitch
 EnvironmentFile=/etc/openvswitch/default.conf
 EnvironmentFile=-/etc/sysconfig/openvswitch
 EnvironmentFile=-/run/openvswitch/useropts
+LimitSTACK=2M
 @begin_dpdk@
 ExecStartPre=-/bin/sh -c '/usr/bin/chown :$${OVS_USER_ID##*:} /dev/hugepages'
 ExecStartPre=-/usr/bin/chmod 0775 /dev/hugepages