| Message ID | 20180130165453.GC26759@altlinux.org |
|---|---|
| State | New |
| Headers | show |
| Series | Tweak test-*exit-race tests to honor the maximum number of processes | expand |
On 01/30/2018 08:54 AM, Dmitry V. Levin wrote: > When the number of threads created by test-*exit-race tests is close > to the maximum number of simultaneous processes per uid, these tests > fail with the following diagnostics: > > error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable > > Workaround this issue by limiting the number of threads created > by these tests to sysconf (_SC_CHILD_MAX) / 2. > > Tested that these tests still fail reliably on x86_64 with glibc-2.26 > and sysconf (_SC_CHILD_MAX) == 512. > > * stdlib/test-atexit-race-common.c: Include <unistd.h>. > (kNumThreads): Remove const qualifier. > (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2. The test should return UNSUPPORTED if an expected minimum number of threads required to reliably trigger the test cannot be created e.g. 512. > --- > ChangeLog | 6 ++++++ > stdlib/test-atexit-race-common.c | 10 ++++++++-- > 2 files changed, 14 insertions(+), 2 deletions(-) > > diff --git a/stdlib/test-atexit-race-common.c b/stdlib/test-atexit-race-common.c > index 4d7f911..7a65264 100644 > --- a/stdlib/test-atexit-race-common.c > +++ b/stdlib/test-atexit-race-common.c > @@ -33,10 +33,11 @@ > > #include <stdio.h> > #include <stdlib.h> > +#include <unistd.h> > #include <support/xthread.h> > > -const size_t kNumThreads = 1024; > -const size_t kNumHandlers = 1024; > +static size_t kNumThreads = 1024; > +static const size_t kNumHandlers = 1024; > > static void * > threadfunc (void *unused) > @@ -62,6 +63,11 @@ do_test (void) > 128KiB for a maximum required VM size of 128MiB. */ > xpthread_attr_setstacksize (&attr, 128 * 1024); > > + long nproc = sysconf (_SC_CHILD_MAX); > + > + if (nproc > 0 && nproc / 2 < kNumThreads) > + kNumThreads = nproc / 2; > + > for (i = 0; i < kNumThreads; ++i) { > xpthread_create (&attr, threadfunc, NULL); > } >
On Tue, Jan 30, 2018 at 12:04:06PM -0800, Carlos O'Donell wrote: > On 01/30/2018 08:54 AM, Dmitry V. Levin wrote: > > When the number of threads created by test-*exit-race tests is close > > to the maximum number of simultaneous processes per uid, these tests > > fail with the following diagnostics: > > > > error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable > > > > Workaround this issue by limiting the number of threads created > > by these tests to sysconf (_SC_CHILD_MAX) / 2. > > > > Tested that these tests still fail reliably on x86_64 with glibc-2.26 > > and sysconf (_SC_CHILD_MAX) == 512. > > > > * stdlib/test-atexit-race-common.c: Include <unistd.h>. > > (kNumThreads): Remove const qualifier. > > (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2. > > The test should return UNSUPPORTED if an expected minimum number of threads > required to reliably trigger the test cannot be created e.g. 512. It depends on what do you call reliably. For example, on an x86_64 box with `getconf _NPROCESSORS_ONLN` == 32 the chance of test-atexit-race *not* to trigger a segfault with glibc-2.26 was about 0.08% with current value of kNumThreads == 1024, and this chance increased roughly 4 times to about 0.34% with kNumThreads == 256.
On 01/30/2018 03:36 PM, Dmitry V. Levin wrote: > On Tue, Jan 30, 2018 at 12:04:06PM -0800, Carlos O'Donell wrote: >> On 01/30/2018 08:54 AM, Dmitry V. Levin wrote: >>> When the number of threads created by test-*exit-race tests is close >>> to the maximum number of simultaneous processes per uid, these tests >>> fail with the following diagnostics: >>> >>> error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable >>> >>> Workaround this issue by limiting the number of threads created >>> by these tests to sysconf (_SC_CHILD_MAX) / 2. >>> >>> Tested that these tests still fail reliably on x86_64 with glibc-2.26 >>> and sysconf (_SC_CHILD_MAX) == 512. >>> >>> * stdlib/test-atexit-race-common.c: Include <unistd.h>. >>> (kNumThreads): Remove const qualifier. >>> (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2. >> >> The test should return UNSUPPORTED if an expected minimum number of threads >> required to reliably trigger the test cannot be created e.g. 512. > > It depends on what do you call reliably. For example, on an x86_64 box > with `getconf _NPROCESSORS_ONLN` == 32 the chance of test-atexit-race > *not* to trigger a segfault with glibc-2.26 was about 0.08% with current > value of kNumThreads == 1024, and this chance increased roughly 4 times > to about 0.34% with kNumThreads == 256. Reliably is > 50% of the time in my opinion. This means that if you sample the test 3-5 times you are almost guaranteed a failure (87.5-97%, multiplicative statistics).
diff --git a/stdlib/test-atexit-race-common.c b/stdlib/test-atexit-race-common.c index 4d7f911..7a65264 100644 --- a/stdlib/test-atexit-race-common.c +++ b/stdlib/test-atexit-race-common.c @@ -33,10 +33,11 @@ #include <stdio.h> #include <stdlib.h> +#include <unistd.h> #include <support/xthread.h> -const size_t kNumThreads = 1024; -const size_t kNumHandlers = 1024; +static size_t kNumThreads = 1024; +static const size_t kNumHandlers = 1024; static void * threadfunc (void *unused) @@ -62,6 +63,11 @@ do_test (void) 128KiB for a maximum required VM size of 128MiB. */ xpthread_attr_setstacksize (&attr, 128 * 1024); + long nproc = sysconf (_SC_CHILD_MAX); + + if (nproc > 0 && nproc / 2 < kNumThreads) + kNumThreads = nproc / 2; + for (i = 0; i < kNumThreads; ++i) { xpthread_create (&attr, threadfunc, NULL); }