diff mbox

autobuild statistics graph

Message ID CAAXf6LVPQVzQpfCdeVrm2yNzEad8j2GS9pnna7YiKVnhTb=MQg@mail.gmail.com
State Not Applicable
Headers show

Commit Message

Thomas De Schampheleire Aug. 29, 2013, 7:40 a.m. UTC
Hi Thomas,

On Wed, Aug 28, 2013 at 10:02 AM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> Hi Thomas,
>
> On Wed, Aug 28, 2013 at 9:48 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Dear Thomas De Schampheleire,
>>
>> On Wed, 28 Aug 2013 09:42:45 +0200, Thomas De Schampheleire wrote:
>>
>>> > Sounds sane. How would you compute what the reference is? Look at the
>>> > build results of the last 30 days (i.e the ones that appear on the
>>> > plot) ?
>>>
>>> I was actually thinking of using the current calculated percentages.
>>> So say on day 1 there are 100 builds, with 70 successful, and on day 2
>>> there have been 50 builds with 25 successful, then I would plot 70%
>>> for the first day and 50% for the second.
>>
>> Ah right, makes sense. The percentages are in fact already shown in the
>> table above.
>>
>>> Given the variation in the configurations that are being built, a
>>> rising success line from one day to the next cannot be interpreted as
>>> an absolute reduction of bugs (could have been 'chance' because the
>>> bad configurations weren't built on the second day) but if you look at
>>> the trend it should tell us something.
>>
>> Right.
>>
>>> Here is some untested code that I think should do the trick (I don't
>>> have a PHP environment at hand here):
>>
>> Your patch is now live at http://autobuild.buildroot.org/stats.php and
>> it seems to work. If you confirm, then I'll commit and push your patch,
>> with you as the author.
>
> That looks good, yes, thanks for testing so quickly.
> One consideration: although the table doesn't have to be larger, the
> 30 days for the graph is somewhat limited. If we want to see a trend,
> then I think a few months can be more interesting. So what about
> changing the SQL query to, say, 180 days (6 months) ?

It seems you already made this change, thanks.

The x axis is now a bit crowded, below untested patch should make it a
bit more readable, could you check?



Best regards,
Thomas

Comments

Thomas Petazzoni Aug. 29, 2013, 8:15 a.m. UTC | #1
Dear Thomas De Schampheleire,

On Thu, 29 Aug 2013 09:40:41 +0200, Thomas De Schampheleire wrote:

> > That looks good, yes, thanks for testing so quickly.
> > One consideration: although the table doesn't have to be larger, the
> > 30 days for the graph is somewhat limited. If we want to see a trend,
> > then I think a few months can be more interesting. So what about
> > changing the SQL query to, say, 180 days (6 months) ?
> 
> It seems you already made this change, thanks.
> 
> The x axis is now a bit crowded, below untested patch should make it a
> bit more readable, could you check?

Yeah, I did some experiment, but as you say the X axis was crowded, and
I didn't had the time to investigate.

> diff --git a/web/graph.php b/web/graph.php
> index 7ac37d8..5585ee5 100644
> --- a/web/graph.php
> +++ b/web/graph.php
> @@ -50,7 +50,7 @@
> $myPicture->setFontProperties(array("FontName"=>"../externals/pchart/fonts/verda
>  /* Define the boundaries of the graph area */
>  $myPicture->setGraphArea(70,50,650,400);
> 
> -$myPicture->drawScale(array("LabelRotation" => 90));
> +$myPicture->drawScale(array("LabelRotation" => 90, "LabelSkip" => 10));
> 
>  $myPicture->drawLegend(20,20,array("Style"=>LEGEND_NOBORDER,"Mode"=>LEGEND_HORIZONTAL));

I'll definitely try this out, thanks!

Another great thing would be to draw some thin vertical lines at the
dates of the various releases (-rc1, -rc2, final and so on), so we can
see if the rate of failures/success correlate with releases or not.
It'd be fun, but I'm not sure it's really worth the effort because
besides the releases themselves, another major thing causes big
variations in the rate of success/failure: when I had a new toolchain
for a new architecture, or a new configuration type (eg. static
builds). I believe those changes in the configuration of the
autobuilders are causing more variations in the results than the
release cycles themselves.

Thanks,

Thomas
Thomas De Schampheleire Aug. 29, 2013, 8:40 a.m. UTC | #2
Hi Thomas,

On Thu, Aug 29, 2013 at 10:15 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

>
> Another great thing would be to draw some thin vertical lines at the
> dates of the various releases (-rc1, -rc2, final and so on), so we can
> see if the rate of failures/success correlate with releases or not.
> It'd be fun, but I'm not sure it's really worth the effort because
> besides the releases themselves, another major thing causes big
> variations in the rate of success/failure: when I had a new toolchain
> for a new architecture, or a new configuration type (eg. static
> builds). I believe those changes in the configuration of the
> autobuilders are causing more variations in the results than the
> release cycles themselves.
>

Yes, I also considered that having the releases indicated in the graph
would be interesting. In fact, if we'd have these big moments shown in
the graph, the actual dates wouldn't really matter anymore and could
be hidden completely IMO.

If you still have the info about when some of the big changes (like
new toolchain etc.) was done, we can definitely put that alongside the
release milestones.

How to best store the milestones (in the database or in the graph.php
file itself), I'm not sure. They'll probably need to be manually
updated, but that could be done both in the database as in graph.php.
How to show these milestones in the graph is also not entirely clear.
I can't find a way to pass some custom x values to  pChart. One
solution is to manipulate the date array from php before passing it to
pChart. This way, if we have a map between date (as it appears in the
database) and event, we can replace the dates by the events, and make
all other strings empty. This should give the desired result I think.
Arnout Vandecappelle Aug. 29, 2013, 3:59 p.m. UTC | #3
On 08/29/13 10:40, Thomas De Schampheleire wrote:
> Hi Thomas,
>
> On Thu, Aug 29, 2013 at 10:15 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>
>>
>> Another great thing would be to draw some thin vertical lines at the
>> dates of the various releases (-rc1, -rc2, final and so on), so we can
>> see if the rate of failures/success correlate with releases or not.
>> It'd be fun, but I'm not sure it's really worth the effort because
>> besides the releases themselves, another major thing causes big
>> variations in the rate of success/failure: when I had a new toolchain
>> for a new architecture, or a new configuration type (eg. static
>> builds). I believe those changes in the configuration of the
>> autobuilders are causing more variations in the results than the
>> release cycles themselves.
>>
>
> Yes, I also considered that having the releases indicated in the graph
> would be interesting. In fact, if we'd have these big moments shown in
> the graph, the actual dates wouldn't really matter anymore and could
> be hidden completely IMO.
>
> If you still have the info about when some of the big changes (like
> new toolchain etc.) was done, we can definitely put that alongside the
> release milestones.
>
> How to best store the milestones (in the database or in the graph.php
> file itself), I'm not sure. They'll probably need to be manually
> updated, but that could be done both in the database as in graph.php.
> How to show these milestones in the graph is also not entirely clear.
> I can't find a way to pass some custom x values to  pChart. One
> solution is to manipulate the date array from php before passing it to
> pChart. This way, if we have a map between date (as it appears in the
> database) and event, we can replace the dates by the events, and make
> all other strings empty. This should give the desired result I think.

  Ideally the horizontal axis should be based on git commit IDs instead 
of build date, and the labels could be the tags.  And you could manually 
add tags to the clone from which the commit IDs are extracted to indicate 
those interesting points where toolchains are added etc. But that would 
be a bit too much effort I guess :-)

  Regards,
  Arnout
Thomas Petazzoni Aug. 30, 2013, 7:47 a.m. UTC | #4
Dear Arnout Vandecappelle,

On Thu, 29 Aug 2013 17:59:09 +0200, Arnout Vandecappelle wrote:

> > Yes, I also considered that having the releases indicated in the graph
> > would be interesting. In fact, if we'd have these big moments shown in
> > the graph, the actual dates wouldn't really matter anymore and could
> > be hidden completely IMO.
> >
> > If you still have the info about when some of the big changes (like
> > new toolchain etc.) was done, we can definitely put that alongside the
> > release milestones.
> >
> > How to best store the milestones (in the database or in the graph.php
> > file itself), I'm not sure. They'll probably need to be manually
> > updated, but that could be done both in the database as in graph.php.
> > How to show these milestones in the graph is also not entirely clear.
> > I can't find a way to pass some custom x values to  pChart. One
> > solution is to manipulate the date array from php before passing it to
> > pChart. This way, if we have a map between date (as it appears in the
> > database) and event, we can replace the dates by the events, and make
> > all other strings empty. This should give the desired result I think.
> 
>   Ideally the horizontal axis should be based on git commit IDs instead 
> of build date, and the labels could be the tags.  And you could manually 
> add tags to the clone from which the commit IDs are extracted to indicate 
> those interesting points where toolchains are added etc. But that would 
> be a bit too much effort I guess :-)

Hum, yeah, I don't mind spending a bit of time on writing/maintaining a
few tools around QA for Buildroot, but I'd like to also keep some time
for real Buildroot development, and not be completely full with stupid
PHP development tasks :-)

Thomas
Thomas Petazzoni Sept. 8, 2013, 1:57 p.m. UTC | #5
Dear Thomas De Schampheleire,

On Thu, 29 Aug 2013 09:40:41 +0200, Thomas De Schampheleire wrote:

> -$myPicture->drawScale(array("LabelRotation" => 90));
> +$myPicture->drawScale(array("LabelRotation" => 90, "LabelSkip" =>
> 10));

Thanks, this works, I've committed this and deployed on autobuild.b.o.

See
http://git.buildroot.net/buildroot-test/commit/?id=51317ea5ce4ea0244744ff284ecfb98a2743520d.

Thanks!

Thomas
diff mbox

Patch

diff --git a/web/graph.php b/web/graph.php
index 7ac37d8..5585ee5 100644
--- a/web/graph.php
+++ b/web/graph.php
@@ -50,7 +50,7 @@ 
$myPicture->setFontProperties(array("FontName"=>"../externals/pchart/fonts/verda
 /* Define the boundaries of the graph area */
 $myPicture->setGraphArea(70,50,650,400);

-$myPicture->drawScale(array("LabelRotation" => 90));
+$myPicture->drawScale(array("LabelRotation" => 90, "LabelSkip" => 10));

 $myPicture->drawLegend(20,20,array("Style"=>LEGEND_NOBORDER,"Mode"=>LEGEND_HORIZONTAL));