scripts/autobuild-run: add --branches option

Message ID 20190115141131.22754-1-arnout@mind.be
State New
Headers show
Series
  • scripts/autobuild-run: add --branches option
Related show

Commit Message

Arnout Vandecappelle Jan. 15, 2019, 2:11 p.m.
From: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>

The autobuild-run script will retrieve the list of branches to test
from autobuild.buildroot.org. However, it the autobuild script is used
to test some local configuration, this is probably not appropriate. The
new --branches option allows to override the URL of the branches CSV
file. A file:/// URL can be used for a local file.

Signed-off-by: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>
---
 scripts/autobuild-run | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Comments

Thomas Petazzoni Jan. 16, 2019, 8:55 a.m. | #1
Hello,

On Tue, 15 Jan 2019 15:11:31 +0100, Arnout Vandecappelle
(Essensium/Mind) wrote:
> From: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>
> 
> The autobuild-run script will retrieve the list of branches to test
> from autobuild.buildroot.org. However, it the autobuild script is used
> to test some local configuration, this is probably not appropriate. The
> new --branches option allows to override the URL of the branches CSV
> file. A file:/// URL can be used for a local file.
> 
> Signed-off-by: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>

I had an alternate patch ready for this, which consists in changing
http-url to be the base URL, and not the submission URL.

I.e:

   '--http-url': 'http://autobuild.buildroot.org/submit/',

would become:

   '--http-url': 'http://autobuild.buildroot.org/',

And so the branches file would be retrieved from $(--http-url)/branches.

Would this work for you, or do you have a set up where the branches
file is not located at the same place as the submission URL ?

Best regards,

Thomas
Arnout Vandecappelle Jan. 16, 2019, 11:58 a.m. | #2
On 16/01/2019 09:55, Thomas Petazzoni wrote:
> Hello,
> 
> On Tue, 15 Jan 2019 15:11:31 +0100, Arnout Vandecappelle
> (Essensium/Mind) wrote:
>> From: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>
>>
>> The autobuild-run script will retrieve the list of branches to test
>> from autobuild.buildroot.org. However, it the autobuild script is used
>> to test some local configuration, this is probably not appropriate. The
>> new --branches option allows to override the URL of the branches CSV
>> file. A file:/// URL can be used for a local file.
>>
>> Signed-off-by: Arnout Vandecappelle <arnout.vandecappelle@essensium.com>
> 
> I had an alternate patch ready for this, which consists in changing
> http-url to be the base URL, and not the submission URL.
> 
> I.e:
> 
>    '--http-url': 'http://autobuild.buildroot.org/submit/',
> 
> would become:
> 
>    '--http-url': 'http://autobuild.buildroot.org/',
> 
> And so the branches file would be retrieved from $(--http-url)/branches.
> 
> Would this work for you, or do you have a set up where the branches
> file is not located at the same place as the submission URL ?

 Yes, that would work as well.

 Note that I'm not actually using the upload option, but since that is gated on
the presence of http-login and http-password, that's fine.

 Regards,
 Arnout

Patch

diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 3d2e99a..77272f5 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -69,6 +69,7 @@  defaults = {
     '--http-url': 'http://autobuild.buildroot.org/submit/',
     '--toolchains-csv': 'support/config-fragments/autobuild/toolchain-configs.csv',
     '--repo': 'https://github.com/buildroot/buildroot.git',
+    '--branches': 'http://autobuild.buildroot.org/branches',
 }
 
 doc = """autobuild-run - run Buildroot autobuilder
@@ -106,6 +107,8 @@  Options:
   --toolchains-csv CSVFILE       Toolchain configuration file
   -r, --repo URL                 URL of Buildroot repository to clone
                                  Defaults to %(--repo)s
+  --branches URL                 URL of CSV file with branches to test
+                                 Defaults to %(--branches)s
 
 Format of the configuration file:
 
@@ -185,7 +188,7 @@  def check_version():
         print("ERROR: script version too old, please upgrade.")
         sys.exit(1)
 
-def get_branch():
+def get_branch(url):
     """Returns the branch that should be built. It fetches a CSV file from
     autobuild.buildroot.org that provides the list of branches to test
     (first field) and their weight (second field). We build a list of
@@ -194,7 +197,7 @@  def get_branch():
     list. This way, branches with a higher weight are more likely to
     be selected.
     """
-    with urlopen_closing('http://autobuild.buildroot.org/branches') as r:
+    with urlopen_closing(url) as r:
         csv_branches = r.readlines()
     branches = []
     for branch in csv.reader(csv_branches):
@@ -314,7 +317,7 @@  def prepare_build(**kwargs):
                   os.path.relpath(f, dldir))
         os.remove(f)
 
-    branch = get_branch()
+    branch = get_branch(kwargs['branches'])
     log_write(log, "INFO: testing branch '%s'" % branch)
 
     # Clone Buildroot. This only happens if the source directory
@@ -764,6 +767,7 @@  def main():
                 nice = (args['--nice'] or 0),
                 toolchains_csv = args['--toolchains-csv'],
                 repo = args['--repo'],
+                branches = args['--branches'],
                 upload = upload,
                 buildpid = buildpid,
                 debug = args['--debug']