| Message ID | 1557872129-9401-1-git-send-email-hzhou8@ebay.com |
|---|---|
| State | Accepted |
| Commit | 4992e00012e768a82764ab24a3091f3b0dfc7e83 |
| Headers | show |
| Series | [ovs-dev] ovn-ctl: Set max open files limit for SB DB. | expand |
On Tue, May 14, 2019 at 3:15 PM Han Zhou <zhouhan@gmail.com> wrote: > From: Han Zhou <hzhou8@ebay.com> > > Increase the limit on the number of open file descriptors, because > SB DB may connect to large number of chassises. > > Signed-off-by: Han Zhou <hzhou8@ebay.com> > --- > ovn/utilities/ovn-ctl | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl > index cca5fac..d348b47 100755 > --- a/ovn/utilities/ovn-ctl > +++ b/ovn/utilities/ovn-ctl > @@ -236,6 +236,14 @@ start_nb_ovsdb() { > } > > start_sb_ovsdb() { > + # Increase the limit on the number of open file descriptors, because > + # SB DB may connect to large number of chassises, on top of > connections > + # for cluster members, northd, and serveral local unix sockets. > + MAXFD=8192 > >why not just change /etc/security/limits instead of doing in code? # Default soft limits before any changes to /etc/security/limits $ ulimit -Sn 1024 $ cat /proc/13055/limits ; where 13055 is ovn sb db pid Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 25231 25231 processes Max open files 1024 1048576 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 25231 25231 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Since ovn nb/sb dbs run as root user, I tried by adding below entries that did the job: $ vim /etc/security/limits.conf root hard nofile 8192 root soft nofile 8192 So next time when you start nb/sb, it will set max open files to 8192. $ cat /proc/14362/limits ;where 14362 is ovn sb db pid Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 25231 25231 processes Max open files 8192 8192 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 25231 25231 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us > + if [ $(ulimit -n) -lt $MAXFD ]; then > + ulimit -n $MAXFD > + fi > + > start_ovsdb__ SB sb OVN_Southbound SB_Global > } > > -- > 2.1.0 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
On Wed, May 15, 2019 at 12:53 AM aginwala <aginwala@asu.edu> wrote: > > > On Tue, May 14, 2019 at 3:15 PM Han Zhou <zhouhan@gmail.com> wrote: > >> From: Han Zhou <hzhou8@ebay.com> >> >> Increase the limit on the number of open file descriptors, because >> SB DB may connect to large number of chassises. >> >> Signed-off-by: Han Zhou <hzhou8@ebay.com> >> --- >> ovn/utilities/ovn-ctl | 8 ++++++++ >> 1 file changed, 8 insertions(+) >> >> diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl >> index cca5fac..d348b47 100755 >> --- a/ovn/utilities/ovn-ctl >> +++ b/ovn/utilities/ovn-ctl >> @@ -236,6 +236,14 @@ start_nb_ovsdb() { >> } >> >> start_sb_ovsdb() { >> + # Increase the limit on the number of open file descriptors, because >> + # SB DB may connect to large number of chassises, on top of >> connections >> + # for cluster members, northd, and serveral local unix sockets. >> + MAXFD=8192 >> > >why not just change /etc/security/limits instead of doing in code? > # Default soft limits before any changes to /etc/security/limits > $ ulimit -Sn > 1024 > $ cat /proc/13055/limits ; where 13055 is ovn sb db pid > Limit Soft Limit Hard Limit Units > Max cpu time unlimited unlimited seconds > Max file size unlimited unlimited bytes > Max data size unlimited unlimited bytes > Max stack size 8388608 unlimited bytes > Max core file size unlimited unlimited bytes > Max resident set unlimited unlimited bytes > Max processes 25231 25231 > processes > Max open files 1024 1048576 files > Max locked memory 65536 65536 bytes > Max address space unlimited unlimited bytes > Max file locks unlimited unlimited locks > Max pending signals 25231 25231 signals > Max msgqueue size 819200 819200 bytes > Max nice priority 0 0 > Max realtime priority 0 0 > Max realtime timeout unlimited unlimited us > > Since ovn nb/sb dbs run as root user, I tried by adding below entries that > did the job: > $ vim /etc/security/limits.conf > root hard nofile 8192 > root soft nofile 8192 > > So next time when you start nb/sb, it will set max open files to 8192. > $ cat /proc/14362/limits ;where 14362 is ovn sb db pid > Limit Soft Limit Hard Limit Units > Max cpu time unlimited unlimited seconds > Max file size unlimited unlimited bytes > Max data size unlimited unlimited bytes > Max stack size 8388608 unlimited bytes > Max core file size unlimited unlimited bytes > Max resident set unlimited unlimited bytes > Max processes 25231 25231 > processes > Max open files 8192 8192 files > Max locked memory 65536 65536 bytes > Max address space unlimited unlimited bytes > Max file locks unlimited unlimited locks > Max pending signals 25231 25231 signals > Max msgqueue size 819200 819200 bytes > Max nice priority 0 0 > Max realtime priority 0 0 > Max realtime timeout unlimited unlimited us > >> + if [ $(ulimit -n) -lt $MAXFD ]; then >> + ulimit -n $MAXFD >> + fi >> + >> start_ovsdb__ SB sb OVN_Southbound SB_Global >> } >> >> -- >> 2.1.0 >> >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > Firstly, settings in /etc/security/limits doesn't work for services started by systemd. (so it doesn't work either for services started by pacemaker if pacemaker is started by systemd). Secondly, even if we can configure systemd settings, which is the current way we are doing, I think it is better to set at the process level directly in ovn-ctl, instead of external system wide settings. For ovs-vswitch it has been done similarly in ovs-ctl.
Acked-by: aginwala <aginwala@ebay.com> On Wed, May 15, 2019 at 8:58 AM Han Zhou <zhouhan@gmail.com> wrote: > > > On Wed, May 15, 2019 at 12:53 AM aginwala <aginwala@asu.edu> wrote: > >> >> >> On Tue, May 14, 2019 at 3:15 PM Han Zhou <zhouhan@gmail.com> wrote: >> >>> From: Han Zhou <hzhou8@ebay.com> >>> >>> Increase the limit on the number of open file descriptors, because >>> SB DB may connect to large number of chassises. >>> >>> Signed-off-by: Han Zhou <hzhou8@ebay.com> >>> --- >>> ovn/utilities/ovn-ctl | 8 ++++++++ >>> 1 file changed, 8 insertions(+) >>> >>> diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl >>> index cca5fac..d348b47 100755 >>> --- a/ovn/utilities/ovn-ctl >>> +++ b/ovn/utilities/ovn-ctl >>> @@ -236,6 +236,14 @@ start_nb_ovsdb() { >>> } >>> >>> start_sb_ovsdb() { >>> + # Increase the limit on the number of open file descriptors, because >>> + # SB DB may connect to large number of chassises, on top of >>> connections >>> + # for cluster members, northd, and serveral local unix sockets. >>> + MAXFD=8192 >>> >> >why not just change /etc/security/limits instead of doing in code? >> # Default soft limits before any changes to /etc/security/limits >> $ ulimit -Sn >> 1024 >> $ cat /proc/13055/limits ; where 13055 is ovn sb db pid >> Limit Soft Limit Hard Limit Units >> Max cpu time unlimited unlimited >> seconds >> Max file size unlimited unlimited bytes >> Max data size unlimited unlimited bytes >> Max stack size 8388608 unlimited bytes >> Max core file size unlimited unlimited bytes >> Max resident set unlimited unlimited bytes >> Max processes 25231 25231 >> processes >> Max open files 1024 1048576 files >> Max locked memory 65536 65536 bytes >> Max address space unlimited unlimited bytes >> Max file locks unlimited unlimited locks >> Max pending signals 25231 25231 >> signals >> Max msgqueue size 819200 819200 bytes >> Max nice priority 0 0 >> Max realtime priority 0 0 >> Max realtime timeout unlimited unlimited us >> >> Since ovn nb/sb dbs run as root user, I tried by adding below entries >> that did the job: >> $ vim /etc/security/limits.conf >> root hard nofile 8192 >> root soft nofile 8192 >> >> So next time when you start nb/sb, it will set max open files to 8192. >> $ cat /proc/14362/limits ;where 14362 is ovn sb db pid >> Limit Soft Limit Hard Limit Units >> Max cpu time unlimited unlimited >> seconds >> Max file size unlimited unlimited bytes >> Max data size unlimited unlimited bytes >> Max stack size 8388608 unlimited bytes >> Max core file size unlimited unlimited bytes >> Max resident set unlimited unlimited bytes >> Max processes 25231 25231 >> processes >> Max open files 8192 8192 files >> Max locked memory 65536 65536 bytes >> Max address space unlimited unlimited bytes >> Max file locks unlimited unlimited locks >> Max pending signals 25231 25231 >> signals >> Max msgqueue size 819200 819200 bytes >> Max nice priority 0 0 >> Max realtime priority 0 0 >> Max realtime timeout unlimited unlimited us >> >>> + if [ $(ulimit -n) -lt $MAXFD ]; then >>> + ulimit -n $MAXFD >>> + fi >>> + >>> start_ovsdb__ SB sb OVN_Southbound SB_Global >>> } >>> >>> -- >>> 2.1.0 >>> >>> _______________________________________________ >>> dev mailing list >>> dev@openvswitch.org >>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> >> > Firstly, settings in /etc/security/limits doesn't work for services > started by systemd. (so it doesn't work either for services started by > pacemaker if pacemaker is started by systemd). > > Secondly, even if we can configure systemd settings, which is the current > way we are doing, I think it is better to set at the process level directly > in ovn-ctl, instead of external system wide settings. For ovs-vswitch it > has been done similarly in ovs-ctl. > >> Yes. This makes sense. Thanks for pointing that ovs-ctl uses the same. I did verify that it uses MAXFD as 65535 when we start with ovs-ctl start.
On Tue, May 14, 2019 at 03:15:29PM -0700, Han Zhou wrote: > From: Han Zhou <hzhou8@ebay.com> > > Increase the limit on the number of open file descriptors, because > SB DB may connect to large number of chassises. > > Signed-off-by: Han Zhou <hzhou8@ebay.com> Applied, thanks!
diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl index cca5fac..d348b47 100755 --- a/ovn/utilities/ovn-ctl +++ b/ovn/utilities/ovn-ctl @@ -236,6 +236,14 @@ start_nb_ovsdb() { } start_sb_ovsdb() { + # Increase the limit on the number of open file descriptors, because + # SB DB may connect to large number of chassises, on top of connections + # for cluster members, northd, and serveral local unix sockets. + MAXFD=8192 + if [ $(ulimit -n) -lt $MAXFD ]; then + ulimit -n $MAXFD + fi + start_ovsdb__ SB sb OVN_Southbound SB_Global }