From patchwork Sun Jul 7 05:28:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atharva Lele X-Patchwork-Id: 1128571 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lYCTL7Xj"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45hHGJ6ysjz9s8m for ; Sun, 7 Jul 2019 15:30:00 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7E9EF85725; Sun, 7 Jul 2019 05:29:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SQhdRQpGoCcW; Sun, 7 Jul 2019 05:29:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 97F088575A; Sun, 7 Jul 2019 05:29:52 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 7A3201BF5DD for ; Sun, 7 Jul 2019 05:29:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7720284AF4 for ; Sun, 7 Jul 2019 05:29:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KsTV2JHYo4WH for ; Sun, 7 Jul 2019 05:29:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by fraxinus.osuosl.org (Postfix) with ESMTPS id A40D484506 for ; Sun, 7 Jul 2019 05:29:49 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id t14so3438497plr.11 for ; Sat, 06 Jul 2019 22:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qa9i9ZHSAtB4trGEnxaO3r8utSZeromPfcjGLjBeKJk=; b=lYCTL7XjMA+e3e2mwS3aDvw+MC/L0TeeOTX/lIPiCv2CMq42jC/5Uxlay5eBTM/gNY RzLuGe/1gdTXJ+c4Xx/vT4VWEFgWvkKobtr9x24/yiXkVCFr/t0NwbgWihzCCWjyGk6o po6ALW8RvXkccAxhj0N40gajvzbg6EPpJdzakgy/Sq71bDBSw2C9hS+KUewwc6Oy9XL2 6LWoggYVmlOiAvGFcmdwrFeQDnE7wn4yuWeIf+X2UCf68YOJJCcqmS7A+fGWyy4A6s/R IXaZOd1nt+RUZqp4gsNgPkscnx5gLKLKtozltaiZYFt5d4Kg/jMT1R2ADcLKWd5vku2X jtaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qa9i9ZHSAtB4trGEnxaO3r8utSZeromPfcjGLjBeKJk=; b=k3BFlKQ87gP6LVk2ej/zIEkNncCpo+cV9lt7iPi0Yit5l78PvLMckN/EZf/zPH/xuH rAxuC7+onqyXiZ8TUG2D6yTEO77i/gwZ3l2oMvwDDA/MBfhPQq4zenw+RHKf4aDgMPOb vOG/x5GVeTJp9gDE4OExC1nZqG0GW0BN3kC8NtGjlSy4Hok7hqsxdntkcVSjb3gdpOsr jGUgt0+402VCy/OhyxxIfVP1n3c2HL8B0h0ACU5EwmfqTRrYLWlXtD9ysb1h1jsYoRfL c0RLhGg/TjA00mFTeO13X2E58wx2nGOUqFIOQzxLGe1MEpYNtptGG/8k4kUAU8PhahdI 1qXA== X-Gm-Message-State: APjAAAWAG/2slkJbdQQNKHOw6NAsOr71U80lrAHy3mB4zzzDt9J7+Oj2 eMR7bHMkNfBj6/8DK9iyjVcIid158I0cOw== X-Google-Smtp-Source: APXvYqzg0v6NEzqnyfv7KwmP7Jne/3CoFZjN+wWQS6F9dBXZb2Fc8bu0gBYKa0YtxN0szsHnLu6pvw== X-Received: by 2002:a17:902:e282:: with SMTP id cf2mr15220418plb.301.1562477388895; Sat, 06 Jul 2019 22:29:48 -0700 (PDT) Received: from localhost.localdomain ([123.201.194.71]) by smtp.gmail.com with ESMTPSA id j24sm18039214pgg.86.2019.07.06.22.29.46 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sat, 06 Jul 2019 22:29:48 -0700 (PDT) From: Atharva Lele To: buildroot@buildroot.org Date: Sun, 7 Jul 2019 10:58:23 +0530 Message-Id: <20190707052831.9469-23-itsatharva@gmail.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190707052831.9469-1-itsatharva@gmail.com> References: <20190707052831.9469-1-itsatharva@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v3 23/31] scripts/autobuild-run: make the HTTP URL really configurable X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atharva Lele , yann.morin.1998@free.fr, thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas Petazzoni The --http-url option allowed to customize the URL at which build results are submitted. However, there were two other places in the script where autobuild.buildroot.org was hardcoded: when checking the script version, and when getting the list of branches to build. This commit changes the --http-url to be the base URL of the autobuild server, and it is used everywhere instead of hardcoding autobuild.buildroot.org. Note: users of autobuild-run that were passing a custom --http-url option such as http://foo.com/submit/ should change it to just http://foo.com/ Signed-off-by: Thomas Petazzoni [Atharva: Adapted Thomas' patches to work with Builder class] Signed-off-by: Atharva Lele Reviewed-by: Arnout Vandecappelle (Essensium/Mind) --- Changes v1 -> v2: - Incorporate Thomas' patch into Builder class series - Move patch for appropriate order of patches --- scripts/autobuild-run | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index ccfa9c4..678bac5 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -66,7 +66,7 @@ defaults = { '--make-opts': '', '--nice': 0, '--pid-file': '/tmp/buildroot-autobuild.pid', - '--http-url': 'http://autobuild.buildroot.org/submit/', + '--http-url': 'http://autobuild.buildroot.org/', '--toolchains-csv': 'support/config-fragments/autobuild/toolchain-configs.csv', '--repo': 'https://github.com/buildroot/buildroot.git', } @@ -145,6 +145,7 @@ from distutils.version import StrictVersion import platform from threading import Thread, Event import datetime +import urlparse if sys.hexversion >= 0x3000000: import configparser @@ -283,7 +284,7 @@ class Builder: self.log = open(os.path.join(self.idir, "instance.log"), "a+") def check_version(self): - with urlopen_closing('http://autobuild.buildroot.org/version') as r: + with urlopen_closing(urlparse.urljoin(self.http_url, 'version')) as r: version = int(decode_bytes(r.readline()).strip()) if version > VERSION: print("ERROR: script version too old, please upgrade.") @@ -291,14 +292,14 @@ class Builder: def get_branch(self): """Returns the branch that should be built. It fetches a CSV file from - autobuild.buildroot.org that provides the list of branches to test + the autobuild server that provides the list of branches to test (first field) and their weight (second field). We build a list of branches, where the branch name is repeated N times, with N being the weight of the branch. We then pick a random branch in this list. This way, branches with a higher weight are more likely to be selected. """ - with urlopen_closing('http://autobuild.buildroot.org/branches') as r: + with urlopen_closing(urlparse.urljoin(self.http_url, 'branches')) as r: csv_branches = r.readlines() branches = [] for branch in csv.reader(csv_branches): @@ -678,7 +679,7 @@ class Builder: "-H", "Expect:", "-F", "uploadedfile=@%s" % os.path.join(self.outputdir, "results.tar.bz2"), "-F", "uploadsubmit=1", - self.http_url], + urlparse.urljoin(self.http_url, 'submit/')], stdout=self.log, stderr=self.log) if ret != 0: log_write(self.log, "INFO: results could not be submitted, %d" % ret)