| Message ID | 20190617093456.6110-1-itsatharva@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [v5,1/4] autobuild-run: move creation of result directory to run_instance() | expand |
Hello, On Mon, 17 Jun 2019 15:04:53 +0530 Atharva Lele <itsatharva@gmail.com> wrote: > We need the result directory to be present when the check_reproducibility() > function (implemented in the next patch) executes. As of now the results > directory is created in the send_results() function which executes after > check_reproducibility() will. > > Signed-off-by: Atharva Lele <itsatharva@gmail.com> > Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr> > > --- > Changes v4: Add new patch to series > --- > scripts/autobuild-run | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) I have applied the series to buildroot-test, thanks! One thing I was not a big fan of was PATCH 1/4, perhaps the check_reproducibility() should have stored its result in some other place, and keep the send_result() function to collect all the result artifacts into the results/ folder. But since it's a minor detail and we want to move forward with this, I applied as-is. I have just restarted the Bootlin autobuilder machine with this new version of autobuild-run, we'll see what happens :-) Thanks! Thomas
Hello Atharva,
On Mon, 17 Jun 2019 20:06:09 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> I have applied the series to buildroot-test, thanks!
So, the first results in the autobuilders with this applied have
appeared:
http://autobuild.buildroot.net/results/0f3/0f35347a0d4b29fa7b657f42eb3d4d176ad75410/
http://autobuild.buildroot.net/results/128/128675664dbce8f35e5a710e85791708263a92f8/
I see two problems:
- The output in build-end.log does not show any indication that the
overall build failure is caused by the build being non-reproducible.
What build-end.log shows is just a "make legal-info" that ends up
successfully, which is very confusing since the build status is
"failure", but the log doesn't show any sort of failure.
I think the check_reproducibility() function should at least show
something in the build log.
- The reason of the failures, visible at
http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK
are just "unknown". We really want to have a better "reason". I
guess the easiest here would be to move the calculation of the
"reason" into autobuild-run instead of having it server-side, as
Arnout has suggested in a separate discussion yesterday.
Best regards,
Thomas
On 18/06/2019 09:14, Thomas Petazzoni wrote: > Hello Atharva, > > On Mon, 17 Jun 2019 20:06:09 +0200 > Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > >> I have applied the series to buildroot-test, thanks! > > So, the first results in the autobuilders with this applied have > appeared: > > http://autobuild.buildroot.net/results/0f3/0f35347a0d4b29fa7b657f42eb3d4d176ad75410/ > http://autobuild.buildroot.net/results/128/128675664dbce8f35e5a710e85791708263a92f8/ > > I see two problems: > > - The output in build-end.log does not show any indication that the > overall build failure is caused by the build being non-reproducible. > What build-end.log shows is just a "make legal-info" that ends up > successfully, which is very confusing since the build status is > "failure", but the log doesn't show any sort of failure. > > I think the check_reproducibility() function should at least show > something in the build log. You can see it through the existence of the "reproducible_results" file. But it would indeed be a good idea to use that file as the build log instead of the normal log. Atharva? > - The reason of the failures, visible at > http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK > are just "unknown". We really want to have a better "reason". I > guess the easiest here would be to move the calculation of the > "reason" into autobuild-run instead of having it server-side, as > Arnout has suggested in a separate discussion yesterday. Ack, that was the plan. The "unkown" is exactly what we expected. I think it's OK for a few weeks. Regards, Arnout
On Tue, 18 Jun 2019 09:34:54 +0200 Arnout Vandecappelle <arnout@mind.be> wrote: > > - The output in build-end.log does not show any indication that the > > overall build failure is caused by the build being non-reproducible. > > What build-end.log shows is just a "make legal-info" that ends up > > successfully, which is very confusing since the build status is > > "failure", but the log doesn't show any sort of failure. > > > > I think the check_reproducibility() function should at least show > > something in the build log. > > You can see it through the existence of the "reproducible_results" file. Yes, I know, I have read the autobuild-run patches before applying them :-) Still the immediate thing you click on when you see a failure is the build log, and when there's no apparent failure it's confusing. > But it would indeed be a good idea to use that file as the build log instead of > the normal log. Correct. > > - The reason of the failures, visible at > > http://autobuild.buildroot.net/?submitter=Thomas+Petazzoni+(Bootlin+server)&status=NOK > > are just "unknown". We really want to have a better "reason". I > > guess the easiest here would be to move the calculation of the > > "reason" into autobuild-run instead of having it server-side, as > > Arnout has suggested in a separate discussion yesterday. > > Ack, that was the plan. The "unkown" is exactly what we expected. I think it's > OK for a few weeks. Yes, it is OK for now, but it would be good to work on fixing this. Best regards, Thomas
On Tue, Jun 18, 2019 at 1:04 PM Arnout Vandecappelle <arnout@mind.be> wrote: > > But it would indeed be a good idea to use that file as the build log > instead of > the normal log. > > Atharva? > Yes, that'll be what I work on once I finish the builder-class. Is that OK?
diff --git a/scripts/autobuild-run b/scripts/autobuild-run index ef2f2a5..190a254 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -470,8 +470,6 @@ def send_results(result, **kwargs): srcdir = os.path.join(idir, "buildroot") resultdir = os.path.join(outputdir, "results") - os.mkdir(resultdir) - shutil.copyfile(os.path.join(outputdir, ".config"), os.path.join(resultdir, "config")) shutil.copyfile(os.path.join(outputdir, "defconfig"), @@ -647,6 +645,9 @@ def run_instance(**kwargs): if ret != 0: continue + resultdir = os.path.join(idir, "output", "results") + os.mkdir(resultdir) + ret = gen_config(**kwargs) if ret != 0: log_write(kwargs['log'], "WARN: failed to generate configuration")