| Message ID | 20201021145859.11201-19-vsementsov@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | preallocate filter | expand |
On 21.10.20 16:58, Vladimir Sementsov-Ogievskiy wrote: > Move to generic format for floats and percentage for error. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > scripts/simplebench/results_to_text.py | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/scripts/simplebench/results_to_text.py b/scripts/simplebench/results_to_text.py > index 58d909ffd9..479f7ac1d4 100644 > --- a/scripts/simplebench/results_to_text.py > +++ b/scripts/simplebench/results_to_text.py > @@ -16,11 +16,22 @@ > # along with this program. If not, see <http://www.gnu.org/licenses/>. > # > > +import math > + > + > +def format_value(x, stdev): > + stdev_pr = stdev / x * 100 > + if stdev_pr < 1.5: > + # don't care too much > + return f'{x:.2g}' > + else: > + return f'{x:.2g} ± {math.ceil(stdev_pr)}%' OK, so no magnitude-based precision this time (except for the %f -> %g change). Works for me. Other than that, I personally don’t like the relative standard deviation much, because the absolute SD immediately shows the 68 % boundaries (and an idea on the 95 % boundaries with 2σ), whereas the RSD just gives an impression on how spread out the data is. (Which I find the absolute SD also does, when given together with the average, which is the case here.) To be completely honest, though, I didn’t even know the term “relative SD” existed until a couple of minutes ago, and I didn’t know it was something that was used at all. And if I haven’t seen the RSD used in practice, I can’t confidently say that I have good reasons not to like it. But, well, I can’t have any confidence in liking it either, and because the change from ASD to RSD is basically the most important change of this patch (which I can’t really agree is an improvement), I can’t really give an R-b. Perhaps this is OK: Acked-by: Max Reitz <mreitz@redhat.com> > > def result_to_text(result): > """Return text representation of bench_one() returned dict.""" > if 'average' in result: > - s = '{:.2f} +- {:.2f}'.format(result['average'], result['stdev']) > + s = format_value(result['average'], result['stdev']) > if 'n-failed' in result: > s += '\n({} failed)'.format(result['n-failed']) > return s >
13.11.2020 18:59, Max Reitz wrote: > On 21.10.20 16:58, Vladimir Sementsov-Ogievskiy wrote: >> Move to generic format for floats and percentage for error. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> --- >> scripts/simplebench/results_to_text.py | 13 ++++++++++++- >> 1 file changed, 12 insertions(+), 1 deletion(-) >> >> diff --git a/scripts/simplebench/results_to_text.py b/scripts/simplebench/results_to_text.py >> index 58d909ffd9..479f7ac1d4 100644 >> --- a/scripts/simplebench/results_to_text.py >> +++ b/scripts/simplebench/results_to_text.py >> @@ -16,11 +16,22 @@ >> # along with this program. If not, see <http://www.gnu.org/licenses/>. >> # >> +import math >> + >> + >> +def format_value(x, stdev): >> + stdev_pr = stdev / x * 100 >> + if stdev_pr < 1.5: >> + # don't care too much >> + return f'{x:.2g}' >> + else: >> + return f'{x:.2g} ± {math.ceil(stdev_pr)}%' > > OK, so no magnitude-based precision this time (except for the %f -> %g change). Works for me. > > Other than that, I personally don’t like the relative standard deviation much, because the absolute SD immediately shows the 68 % boundaries (and an idea on the 95 % boundaries with 2σ), whereas the RSD just gives an impression on how spread out the data is. (Which I find the absolute SD also does, when given together with the average, which is the case here.) I realized that for myself, the only thing I need from +-deviation is a kind of "reliability" of the result. And it's more obvious for me in percentage. Looking at the table with a lot of numbers, where most of values have deviation less than +-5%, some values with deviation +-30 would catch the eye immediately. And with absolute SD I have to look through the table more carefully. > > To be completely honest, though, I didn’t even know the term “relative SD” existed until a couple of minutes ago, and I didn’t know it was something that was used at all. > And if I haven’t seen the RSD used in practice, I can’t confidently say that I have good reasons not to like it. > > But, well, I can’t have any confidence in liking it either, and because the change from ASD to RSD is basically the most important change of this patch (which I can’t really agree is an improvement), I can’t really give an R-b. > > Perhaps this is OK: > > Acked-by: Max Reitz <mreitz@redhat.com> > Thanks! I don't have strict opinion about how this should look. For now these scripts don't have wide usage... We can always add an option to adjust table view, and I will not mind making absolute SD the default view.
diff --git a/scripts/simplebench/results_to_text.py b/scripts/simplebench/results_to_text.py index 58d909ffd9..479f7ac1d4 100644 --- a/scripts/simplebench/results_to_text.py +++ b/scripts/simplebench/results_to_text.py @@ -16,11 +16,22 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # +import math + + +def format_value(x, stdev): + stdev_pr = stdev / x * 100 + if stdev_pr < 1.5: + # don't care too much + return f'{x:.2g}' + else: + return f'{x:.2g} ± {math.ceil(stdev_pr)}%' + def result_to_text(result): """Return text representation of bench_one() returned dict.""" if 'average' in result: - s = '{:.2f} +- {:.2f}'.format(result['average'], result['stdev']) + s = format_value(result['average'], result['stdev']) if 'n-failed' in result: s += '\n({} failed)'.format(result['n-failed']) return s
Move to generic format for floats and percentage for error. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> --- scripts/simplebench/results_to_text.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)