| Message ID | 20170406181854.5242-8-arnout@mind.be |
|---|---|
| State | Superseded |
| Headers | show |
Hello, On Thu, 6 Apr 2017 20:18:48 +0200, Arnout Vandecappelle (Essensium/Mind) wrote: > Now we have the toolchain configs in the buildroot directory itself, it > is no longer necessary to fetch it from the toolchain URL. > > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> > Cc: Yann E. MORIN <yann.morin.1998@free.fr> On the principle, I am OK with this direction, but it is needs to be coordinated with a change to autobuild-run to also use the base autobuild defconfigs from the Buildroot tree. Otherwise, we will have two copies: one used by people running test-pkg, one used by the actual autobuilders. It is not difficult to see those two copies quickly getting out of sync. Best regards, Thomas
Thomas, All, On 2017-04-06 21:37 +0200, Thomas Petazzoni spake thusly: > On Thu, 6 Apr 2017 20:18:48 +0200, Arnout Vandecappelle > (Essensium/Mind) wrote: > > Now we have the toolchain configs in the buildroot directory itself, it > > is no longer necessary to fetch it from the toolchain URL. > > > > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> > > Cc: Yann E. MORIN <yann.morin.1998@free.fr> > > On the principle, I am OK with this direction, but it is needs to be > coordinated with a change to autobuild-run to also use the base > autobuild defconfigs from the Buildroot tree. Otherwise, we will have > two copies: one used by people running test-pkg, one used by the actual > autobuilders. It is not difficult to see those two copies quickly > getting out of sync. I think the idea is not to sync, but to get rid of the toolchain configs in autobuild-run altogether, and only use those in the Buildroot tree. Regards, Yann E. MORIN.
Hello, On Thu, 6 Apr 2017 22:48:04 +0200, Yann E. MORIN wrote: > > On the principle, I am OK with this direction, but it is needs to be > > coordinated with a change to autobuild-run to also use the base > > autobuild defconfigs from the Buildroot tree. Otherwise, we will have > > two copies: one used by people running test-pkg, one used by the actual > > autobuilders. It is not difficult to see those two copies quickly > > getting out of sync. > > I think the idea is not to sync, but to get rid of the toolchain configs > in autobuild-run altogether, and only use those in the Buildroot tree. Huh, you must have misunderstood what I wrote, or it wasn't clear. My point was precisely that we want to avoid sync'ing them. But to achieve that the autobuild-run script needs to be changed to use base toolchain configs from the Buildroot tree rather than through the CSV file downloaded from autobuild.b.o. And I'm exactly saying that we need to change the autobuild-run script at the same time as we move base toolchain configs in the tree, otherwise we have duplication. Thomas
On 06-04-17 22:50, Thomas Petazzoni wrote: > Hello, > > On Thu, 6 Apr 2017 22:48:04 +0200, Yann E. MORIN wrote: > >>> On the principle, I am OK with this direction, but it is needs to be >>> coordinated with a change to autobuild-run to also use the base >>> autobuild defconfigs from the Buildroot tree. Otherwise, we will have >>> two copies: one used by people running test-pkg, one used by the actual >>> autobuilders. It is not difficult to see those two copies quickly >>> getting out of sync. >> >> I think the idea is not to sync, but to get rid of the toolchain configs >> in autobuild-run altogether, and only use those in the Buildroot tree. > > Huh, you must have misunderstood what I wrote, or it wasn't clear. > > My point was precisely that we want to avoid sync'ing them. But to > achieve that the autobuild-run script needs to be changed to use base > toolchain configs from the Buildroot tree rather than through the > CSV file downloaded from autobuild.b.o. And I'm exactly saying that we > need to change the autobuild-run script at the same time as we move > base toolchain configs in the tree, otherwise we have duplication. I'm working on updating autobuild-run. Unfortunately it's a bit difficult to test changes in that script, e.g. removing the condition on BR2_STATIC_LIBS. I basically have to wait until the script hits a glibc toolchain to see if it works correctly. Can you think of any way to speed that up? Regards, Arnout
Hello, On Fri, 7 Apr 2017 12:35:09 +0200, Arnout Vandecappelle wrote: > I'm working on updating autobuild-run. Unfortunately it's a bit difficult to > test changes in that script, e.g. removing the condition on BR2_STATIC_LIBS. I > basically have to wait until the script hits a glibc toolchain to see if it > works correctly. Can you think of any way to speed that up? For this sort of situation, I was editing the CSV file to keep only the one toolchain I'm interested in. Or alternatively, hacking the function that chooses the toolchain to always pick up the one I'm interested in. Thomas
diff --git a/support/scripts/test-pkg b/support/scripts/test-pkg index 67f88ec380..f86d266e6a 100755 --- a/support/scripts/test-pkg +++ b/support/scripts/test-pkg @@ -1,7 +1,7 @@ #!/bin/bash set -e -TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv' +TOOLCHAINS_DIR=configs/autobuild main() { local o O opts @@ -47,11 +47,10 @@ main() { dir="${HOME}/br-test-pkg" fi - # Extract the URLs of the toolchains; drop internal toolchains - # E.g.: http://server/path/to/name.config,arch,libc - # --> http://server/path/to/name.config - toolchains=($(curl -s "${TOOLCHAINS_URL}" \ - |sed -r -e 's/,.*//; /internal/d;' \ + # Find external toolchains + # shellcheck disable=SC2011 + toolchains=($(ls -1 "${TOOLCHAINS_DIR}"/*_defconfig \ + |xargs -r grep -l -F -x "BR2_TOOLCHAIN_EXTERNAL=y" \ |if [ ${random} -gt 0 ]; then \ sort -R |head -n ${random} else @@ -62,7 +61,7 @@ main() { nb_tc="${#toolchains[@]}" if [ ${nb_tc} -eq 0 ]; then - printf "error: no toolchain found (networking issue?)\n" >&2; exit 1 + printf "error: no toolchain found\n" >&2; exit 1 fi nb=0 @@ -72,7 +71,7 @@ main() { for toolchainconfig in "${toolchains[@]}"; do : $((nb++)) # Using basename(1) on a URL works nicely - toolchain="$(basename "${toolchainconfig}" .config)" + toolchain="$(basename "${toolchainconfig}" _defconfig)" build_dir="${dir}/${toolchain}" mkdir -p "${build_dir}" printf "%40s [%*d/%d]: " "${toolchain}" ${#nb_tc} ${nb} ${nb_tc} @@ -91,15 +90,11 @@ main() { build_one() { local dir="${1}" - local url="${2}" + local toolchainconfig="${2}" local cfg="${3}" local pkg="${4}" - if ! curl -s "${url}" >"${dir}/.config"; then - return 2 - fi - - cat "support/misc/minimal.config" "${cfg}" >>"${dir}/.config" + cat "${toolchainconfig}" "support/misc/minimal.config" "${cfg}" >>"${dir}/.config" if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then return 2 @@ -151,8 +146,10 @@ In case failures are noticed, you can fix the package and just re-run the same command again; it will re-run the test where it failed. If you did specify a package (with -p), the package build dir will be removed first. -The list of toolchains is retrieved from the Buildroot autobuilders, available -at ${TOOLCHAINS_URL}. +The toolchains to try are retrieved from ${TOOLCHAINS_DIR}. This directory +contains a defconfig file for each toolchain configuration used by the +autobuilders. Only the external toolchains are tried, because building a +Buildroot toolchain would take too long. Options:
Now we have the toolchain configs in the buildroot directory itself, it is no longer necessary to fetch it from the toolchain URL. Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Yann E. MORIN <yann.morin.1998@free.fr> --- support/scripts/test-pkg | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-)