Message ID | 20190626153841.GA7331@bell-sw.com |
---|---|
State | New |
Headers | show |
Series | Bump up the runtime for "short" benchmarks | expand |
On 6/26/19 11:38 AM, Anton Youdkevitch wrote: > Some benchmarks with a very short runtime shows significantly > different results across runs on Aarch64 - up to tens of percents. > Increasing the runtime to 100ms+ makes the deviation under 5%. > > Tested on Aarch64 and x86-64. > > * benchtests/bench-memccpy.c: replace INNER_LOOP_ITERS > with INNER_LOOP_ITERS_LARGE > * benchtests/bench-memchr.c: Likewise > * benchtests/bench-rawmemchr.c: Likewise > * benchtests/bench-strcat.c: Likewise > * benchtests/bench-strchr.c: Likewise > * benchtests/bench-string.h: Likewise > * benchtests/bench-strlen.c: Likewise > * benchtests/bench-strncpy.c: Likewise > * benchtests/bench-strnlen.c: Likewise > > Looks OK? Do you have commit or copyright assignment? OK for master. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
On 6/26/2019 11:32 PM, Carlos O'Donell wrote: > On 6/26/19 11:38 AM, Anton Youdkevitch wrote: >> Some benchmarks with a very short runtime shows significantly >> different results across runs on Aarch64 - up to tens of percents. >> Increasing the runtime to 100ms+ makes the deviation under 5%. >> >> Tested on Aarch64 and x86-64. >> >> * benchtests/bench-memccpy.c: replace INNER_LOOP_ITERS >> with INNER_LOOP_ITERS_LARGE >> * benchtests/bench-memchr.c: Likewise >> * benchtests/bench-rawmemchr.c: Likewise >> * benchtests/bench-strcat.c: Likewise >> * benchtests/bench-strchr.c: Likewise >> * benchtests/bench-string.h: Likewise >> * benchtests/bench-strlen.c: Likewise >> * benchtests/bench-strncpy.c: Likewise >> * benchtests/bench-strnlen.c: Likewise >> >> Looks OK? > Do you have commit or copyright assignment? I don't. Previously my colleague that has did it on my behalf. > > OK for master. > > Reviewed-by: Carlos O'Donell <carlos@redhat.com> >
On 6/27/19 5:33 AM, Anton Youdkevitch wrote: > On 6/26/2019 11:32 PM, Carlos O'Donell wrote: >> On 6/26/19 11:38 AM, Anton Youdkevitch wrote: >>> Some benchmarks with a very short runtime shows significantly >>> different results across runs on Aarch64 - up to tens of percents. >>> Increasing the runtime to 100ms+ makes the deviation under 5%. >>> >>> Tested on Aarch64 and x86-64. >>> >>> * benchtests/bench-memccpy.c: replace INNER_LOOP_ITERS >>> with INNER_LOOP_ITERS_LARGE >>> * benchtests/bench-memchr.c: Likewise >>> * benchtests/bench-rawmemchr.c: Likewise >>> * benchtests/bench-strcat.c: Likewise >>> * benchtests/bench-strchr.c: Likewise >>> * benchtests/bench-string.h: Likewise >>> * benchtests/bench-strlen.c: Likewise >>> * benchtests/bench-strncpy.c: Likewise >>> * benchtests/bench-strnlen.c: Likewise >>> >>> Looks OK? >> Do you have commit or copyright assignment? > I don't. Previously my colleague that has did it on my behalf. The modern FSF assignment process is fairly painless (all digital depending on your country). If you setup a "futures" assignment then we could just review and take all future patches you post. http://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future https://sourceware.org/glibc/wiki/Contribution%20checklist#FSF_copyright_Assignment
From 3e55575cde5905e6550bb33161dfda657283b823 Mon Sep 17 00:00:00 2001 From: Anton Youdkevitch <anton.youdkevitch@bell-sw.com> Date: Wed, 26 Jun 2019 14:33:50 +0000 Subject: [PATCH] Bump up the runtime for "short" benchmarks The runtime of the benchmarks that ran less than 100ms on Aarch64 was increased. The new INNER_LOOP_ITERS_LARGE constant for such benchmaks was introduced. The goal for the deviation being under 5% between runs is achieved. --- benchtests/bench-memccpy.c | 2 +- benchtests/bench-memchr.c | 2 +- benchtests/bench-rawmemchr.c | 2 +- benchtests/bench-strcat.c | 2 +- benchtests/bench-strchr.c | 2 +- benchtests/bench-string.h | 1 + benchtests/bench-strlen.c | 2 +- benchtests/bench-strncpy.c | 2 +- benchtests/bench-strnlen.c | 2 +- 9 files changed, 9 insertions(+), 8 deletions(-) diff --git a/benchtests/bench-memccpy.c b/benchtests/bench-memccpy.c index d3a2df5..46f96d5 100644 --- a/benchtests/bench-memccpy.c +++ b/benchtests/bench-memccpy.c @@ -41,7 +41,7 @@ static void do_one_test (impl_t *impl, void *dst, const void *src, int c, size_t len, size_t n) { - size_t i, iters = INNER_LOOP_ITERS; + size_t i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; TIMING_NOW (start); diff --git a/benchtests/bench-memchr.c b/benchtests/bench-memchr.c index 9bd07eb..8a9909a 100644 --- a/benchtests/bench-memchr.c +++ b/benchtests/bench-memchr.c @@ -56,7 +56,7 @@ SIMPLE_MEMCHR (const CHAR *s, int c, size_t n) static void do_one_test (impl_t *impl, const CHAR *s, int c, size_t n) { - size_t i, iters = INNER_LOOP_ITERS; + size_t i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; TIMING_NOW (start); diff --git a/benchtests/bench-rawmemchr.c b/benchtests/bench-rawmemchr.c index 44f3114..5cdfdf7 100644 --- a/benchtests/bench-rawmemchr.c +++ b/benchtests/bench-rawmemchr.c @@ -39,7 +39,7 @@ IMPL (generic_rawmemchr, 0) static void do_one_test (impl_t *impl, const char *s, int c, char *exp_res) { - size_t i, iters = INNER_LOOP_ITERS; + size_t i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; char *res = CALL (impl, s, c); if (res != exp_res) diff --git a/benchtests/bench-strcat.c b/benchtests/bench-strcat.c index 39d9fef..08841d0 100644 --- a/benchtests/bench-strcat.c +++ b/benchtests/bench-strcat.c @@ -51,7 +51,7 @@ IMPL (generic_strcat, 0) static void do_one_test (impl_t *impl, CHAR *dst, const CHAR *src) { - size_t k = STRLEN (dst), i, iters = INNER_LOOP_ITERS; + size_t k = STRLEN (dst), i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; if (CALL (impl, dst, src) != dst) diff --git a/benchtests/bench-strchr.c b/benchtests/bench-strchr.c index b8dba17..10d06fe 100644 --- a/benchtests/bench-strchr.c +++ b/benchtests/bench-strchr.c @@ -76,7 +76,7 @@ IMPL (STRCHR, 1) static void do_one_test (impl_t *impl, const CHAR *s, int c, const CHAR *exp_res) { - size_t i, iters = INNER_LOOP_ITERS; + size_t i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; TIMING_NOW (start); diff --git a/benchtests/bench-string.h b/benchtests/bench-string.h index e7fb015..7d35791 100644 --- a/benchtests/bench-string.h +++ b/benchtests/bench-string.h @@ -125,6 +125,7 @@ extern impl_t __start_impls[], __stop_impls[]; # define OPT_SEED 10002 # define INNER_LOOP_ITERS 8192 +# define INNER_LOOP_ITERS_LARGE 131072 # define INNER_LOOP_ITERS_MEDIUM 1024 # define INNER_LOOP_ITERS_SMALL 32 diff --git a/benchtests/bench-strlen.c b/benchtests/bench-strlen.c index 47363af..1997ad7 100644 --- a/benchtests/bench-strlen.c +++ b/benchtests/bench-strlen.c @@ -48,7 +48,7 @@ IMPL (STRLEN, 1) static void do_one_test (json_ctx_t *json_ctx, impl_t *impl, const CHAR *s, size_t exp_len) { - size_t len = CALL (impl, s), i, iters = INNER_LOOP_ITERS; + size_t len = CALL (impl, s), i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; if (len != exp_len) diff --git a/benchtests/bench-strncpy.c b/benchtests/bench-strncpy.c index 93bd6d5..c6a6799 100644 --- a/benchtests/bench-strncpy.c +++ b/benchtests/bench-strncpy.c @@ -54,7 +54,7 @@ typedef CHAR *(*proto_t) (CHAR *, const CHAR *, size_t); static void do_one_test (impl_t *impl, CHAR *dst, const CHAR *src, size_t len, size_t n) { - size_t i, iters = INNER_LOOP_ITERS; + size_t i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; if (CALL (impl, dst, src, n) != STRNCPY_RESULT (dst, len, n)) diff --git a/benchtests/bench-strnlen.c b/benchtests/bench-strnlen.c index e9e51b2..72ec303 100644 --- a/benchtests/bench-strnlen.c +++ b/benchtests/bench-strnlen.c @@ -51,7 +51,7 @@ IMPL (generic_strnlen, 0) static void do_one_test (impl_t *impl, const CHAR *s, size_t maxlen, size_t exp_len) { - size_t len = CALL (impl, s, maxlen), i, iters = INNER_LOOP_ITERS; + size_t len = CALL (impl, s, maxlen), i, iters = INNER_LOOP_ITERS_LARGE; timing_t start, stop, cur; if (len != exp_len) -- 2.7.4