Message ID | 20220111153450.912230-1-andre.przywara@arm.com |
---|---|
State | Accepted |
Commit | 5ecdd529ae38fecd35bfc41869058802c804a01e |
Delegated to: | Simon Glass |
Headers | show |
Series | genboardscfg: limit to 240 jobs | expand |
On 1/11/22 16:34, Andre Przywara wrote: > When genboardscfg.py is run on machines with 255 or more cores, the > process will consume more than 1024 file descriptors, which is a common > standard ulimit for user processes. As a consequence it will fail with a > lenghty Python trace, with the almost hidden message: > OSError: [Errno 24] Too many open files > > It's somewhat questionable whether that level of parallelity is actually > useful for genboardscfg, so we limit the *default* number of jobs to the > safe number of 240, to avoid the problem. > If a user persists, she can still force a higher number via the -j Nits: %s/persists/insists/ %s/she/they/ if you don't specifically mean a female user. Otherwise Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de> > parameter - hopefully having raised the ulimit accordingly beforehand. > > Signed-off-by: Andre Przywara <andre.przywara@arm.com> > --- > tools/genboardscfg.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py > index 4ee7aa1f89..07bf681d1d 100755 > --- a/tools/genboardscfg.py > +++ b/tools/genboardscfg.py > @@ -430,7 +430,7 @@ def main(): > # Add options here > parser.add_option('-f', '--force', action="store_true", default=False, > help='regenerate the output even if it is new') > - parser.add_option('-j', '--jobs', type='int', default=cpu_count, > + parser.add_option('-j', '--jobs', type='int', default=min(cpu_count, 240), > help='the number of jobs to run simultaneously') > parser.add_option('-o', '--output', default=OUTPUT_FILE, > help='output file [default=%s]' % OUTPUT_FILE)
On Tue, 11 Jan 2022 at 08:35, Andre Przywara <andre.przywara@arm.com> wrote: > > When genboardscfg.py is run on machines with 255 or more cores, the > process will consume more than 1024 file descriptors, which is a common > standard ulimit for user processes. As a consequence it will fail with a > lenghty Python trace, with the almost hidden message: > OSError: [Errno 24] Too many open files > > It's somewhat questionable whether that level of parallelity is actually > useful for genboardscfg, so we limit the *default* number of jobs to the > safe number of 240, to avoid the problem. > If a user persists, she can still force a higher number via the -j > parameter - hopefully having raised the ulimit accordingly beforehand. > > Signed-off-by: Andre Przywara <andre.przywara@arm.com> > --- > tools/genboardscfg.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Simon Glass <sjg@chromium.org> Please send such machine. Thank you.
On Tue, 11 Jan 2022 at 08:35, Andre Przywara <andre.przywara@arm.com> wrote: > > When genboardscfg.py is run on machines with 255 or more cores, the > process will consume more than 1024 file descriptors, which is a common > standard ulimit for user processes. As a consequence it will fail with a > lenghty Python trace, with the almost hidden message: > OSError: [Errno 24] Too many open files > > It's somewhat questionable whether that level of parallelity is actually > useful for genboardscfg, so we limit the *default* number of jobs to the > safe number of 240, to avoid the problem. > If a user persists, she can still force a higher number via the -j > parameter - hopefully having raised the ulimit accordingly beforehand. > > Signed-off-by: Andre Przywara <andre.przywara@arm.com> > --- > tools/genboardscfg.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Simon Glass <sjg@chromium.org> Please send such machine. Thank you. Applied to u-boot-dm, thanks!
diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py index 4ee7aa1f89..07bf681d1d 100755 --- a/tools/genboardscfg.py +++ b/tools/genboardscfg.py @@ -430,7 +430,7 @@ def main(): # Add options here parser.add_option('-f', '--force', action="store_true", default=False, help='regenerate the output even if it is new') - parser.add_option('-j', '--jobs', type='int', default=cpu_count, + parser.add_option('-j', '--jobs', type='int', default=min(cpu_count, 240), help='the number of jobs to run simultaneously') parser.add_option('-o', '--output', default=OUTPUT_FILE, help='output file [default=%s]' % OUTPUT_FILE)
When genboardscfg.py is run on machines with 255 or more cores, the process will consume more than 1024 file descriptors, which is a common standard ulimit for user processes. As a consequence it will fail with a lenghty Python trace, with the almost hidden message: OSError: [Errno 24] Too many open files It's somewhat questionable whether that level of parallelity is actually useful for genboardscfg, so we limit the *default* number of jobs to the safe number of 240, to avoid the problem. If a user persists, she can still force a higher number via the -j parameter - hopefully having raised the ulimit accordingly beforehand. Signed-off-by: Andre Przywara <andre.przywara@arm.com> --- tools/genboardscfg.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)