[v5,1/4] autobuild-run: move creation of result directory to run_instance()
diff mbox series

Message ID 20190617093456.6110-1-itsatharva@gmail.com
State Accepted
Headers show
Series
  • [v5,1/4] autobuild-run: move creation of result directory to run_instance()
Related show

Commit Message

Atharva Lele June 17, 2019, 9:34 a.m. UTC
We need the result directory to be present when the check_reproducibility()
function (implemented in the next patch) executes. As of now the results
directory is created in the send_results() function which executes after
check_reproducibility() will.

Signed-off-by: Atharva Lele <itsatharva@gmail.com>
Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr>

---
Changes v4: Add new patch to series
---
 scripts/autobuild-run | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Comments

Thomas Petazzoni June 17, 2019, 6:06 p.m. UTC | #1
Hello,

On Mon, 17 Jun 2019 15:04:53 +0530
Atharva Lele <itsatharva@gmail.com> wrote:

> We need the result directory to be present when the check_reproducibility()
> function (implemented in the next patch) executes. As of now the results
> directory is created in the send_results() function which executes after
> check_reproducibility() will.
> 
> Signed-off-by: Atharva Lele <itsatharva@gmail.com>
> Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr>
> 
> ---
> Changes v4: Add new patch to series
> ---
>  scripts/autobuild-run | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

I have applied the series to buildroot-test, thanks!

One thing I was not a big fan of was PATCH 1/4, perhaps the
check_reproducibility() should have stored its result in some other
place, and keep the send_result() function to collect all the result
artifacts into the results/ folder. But since it's a minor detail and
we want to move forward with this, I applied as-is.

I have just restarted the Bootlin autobuilder machine with this new
version of autobuild-run, we'll see what happens :-)

Thanks!

Thomas
Thomas Petazzoni June 18, 2019, 7:14 a.m. UTC | #2
Hello Atharva,

On Mon, 17 Jun 2019 20:06:09 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:

> I have applied the series to buildroot-test, thanks!

So, the first results in the autobuilders with this applied have
appeared:

  http://autobuild.buildroot.net/results/0f3/0f35347a0d4b29fa7b657f42eb3d4d176ad75410/
  http://autobuild.buildroot.net/results/128/128675664dbce8f35e5a710e85791708263a92f8/

I see two problems:

 - The output in build-end.log does not show any indication that the
   overall build failure is caused by the build being non-reproducible.
   What build-end.log shows is just a "make legal-info" that ends up
   successfully, which is very confusing since the build status is
   "failure", but the log doesn't show any sort of failure.

   I think the check_reproducibility() function should at least show
   something in the build log.

 - The reason of the failures, visible at
   http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK
   are just "unknown". We really want to have a better "reason". I
   guess the easiest here would be to move the calculation of the
   "reason" into autobuild-run instead of having it server-side, as
   Arnout has suggested in a separate discussion yesterday.

Best regards,

Thomas
Arnout Vandecappelle June 18, 2019, 7:34 a.m. UTC | #3
On 18/06/2019 09:14, Thomas Petazzoni wrote:
> Hello Atharva,
> 
> On Mon, 17 Jun 2019 20:06:09 +0200
> Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> 
>> I have applied the series to buildroot-test, thanks!
> 
> So, the first results in the autobuilders with this applied have
> appeared:
> 
>   http://autobuild.buildroot.net/results/0f3/0f35347a0d4b29fa7b657f42eb3d4d176ad75410/
>   http://autobuild.buildroot.net/results/128/128675664dbce8f35e5a710e85791708263a92f8/
> 
> I see two problems:
> 
>  - The output in build-end.log does not show any indication that the
>    overall build failure is caused by the build being non-reproducible.
>    What build-end.log shows is just a "make legal-info" that ends up
>    successfully, which is very confusing since the build status is
>    "failure", but the log doesn't show any sort of failure.
> 
>    I think the check_reproducibility() function should at least show
>    something in the build log.

 You can see it through the existence of the "reproducible_results" file.

 But it would indeed be a good idea to use that file as the build log instead of
the normal log.

 Atharva?


>  - The reason of the failures, visible at
>    http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK
>    are just "unknown". We really want to have a better "reason". I
>    guess the easiest here would be to move the calculation of the
>    "reason" into autobuild-run instead of having it server-side, as
>    Arnout has suggested in a separate discussion yesterday.

 Ack, that was the plan. The "unkown" is exactly what we expected. I think it's
OK for a few weeks.

 Regards,
 Arnout
Thomas Petazzoni June 18, 2019, 7:45 a.m. UTC | #4
On Tue, 18 Jun 2019 09:34:54 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> >  - The output in build-end.log does not show any indication that the
> >    overall build failure is caused by the build being non-reproducible.
> >    What build-end.log shows is just a "make legal-info" that ends up
> >    successfully, which is very confusing since the build status is
> >    "failure", but the log doesn't show any sort of failure.
> > 
> >    I think the check_reproducibility() function should at least show
> >    something in the build log.  
> 
>  You can see it through the existence of the "reproducible_results" file.

Yes, I know, I have read the autobuild-run patches before applying
them :-)

Still the immediate thing you click on when you see a failure is the
build log, and when there's no apparent failure it's confusing.

>  But it would indeed be a good idea to use that file as the build log instead of
> the normal log.

Correct.

> >  - The reason of the failures, visible at
> >    http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK
> >    are just "unknown". We really want to have a better "reason". I
> >    guess the easiest here would be to move the calculation of the
> >    "reason" into autobuild-run instead of having it server-side, as
> >    Arnout has suggested in a separate discussion yesterday.  
> 
>  Ack, that was the plan. The "unkown" is exactly what we expected. I think it's
> OK for a few weeks.

Yes, it is OK for now, but it would be good to work on fixing this.

Best regards,

Thomas
Atharva Lele June 18, 2019, 8:25 a.m. UTC | #5
On Tue, Jun 18, 2019 at 1:04 PM Arnout Vandecappelle <arnout@mind.be> wrote:

>
>  But it would indeed be a good idea to use that file as the build log
> instead of
> the normal log.
>
>  Atharva?
>

Yes, that'll be what I work on once I finish the builder-class. Is that OK?

Patch
diff mbox series

diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index ef2f2a5..190a254 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -470,8 +470,6 @@  def send_results(result, **kwargs):
     srcdir = os.path.join(idir, "buildroot")
     resultdir = os.path.join(outputdir, "results")
 
-    os.mkdir(resultdir)
-
     shutil.copyfile(os.path.join(outputdir, ".config"),
                     os.path.join(resultdir, "config"))
     shutil.copyfile(os.path.join(outputdir, "defconfig"),
@@ -647,6 +645,9 @@  def run_instance(**kwargs):
         if ret != 0:
             continue
 
+        resultdir = os.path.join(idir, "output", "results")
+        os.mkdir(resultdir)
+
         ret = gen_config(**kwargs)
         if ret != 0:
             log_write(kwargs['log'], "WARN: failed to generate configuration")