| Message ID | 20240517145347.269607-1-romain.naour@smile.fr |
|---|---|
| State | Accepted |
| Headers | show |
| Series | package/go-bootstrap-stage3/go-bootstrap-stage3: fix go-bootstrap when parent dir contains invalid .git | expand |
Romain, All, On 2024-05-17 16:53 +0200, Romain Naour spake thusly: > Since go-bootstrap-stage3 package has been added to Buildroot [1], all > tests using go are failing only on Gitlab-ci jobs. The build succeed > if we use docker-run script. > > Since the issue is only trigged by Gitlab-ci environment, the issue > is related to the ownership of the (buildroot) git tree, see [2]. > go seems to ignore the .gitconfig set by before_script. > > Anyway, go should not try to use git to retrieve some info since we > are not building it from the go repository. > > go package was already patched for a similar issue when building > host-go within docker [3]. Apply the same patch to go-bootstrap-stage3 > to disable VCS support. > > See > http://lists.busybox.net/pipermail/buildroot/2024-May/691273.html > > [1] f00eb37de9b4b0ddbdeb5c7ebae48c883c27e132 > [2] a016b693f7830f3c8ae815851d3204b8b6e99821 > [3] bc8e70a08ba083d978c828e31442a7645c9099d7 > > Fixes: > https://gitlab.com/buildroot.org/buildroot/-/jobs/6833307649 (tests.fs.test_oci.TestOci) > https://gitlab.com/buildroot.org/buildroot/-/jobs/6833307823 (tests.package.test_docker_compose.TestDockerCompose) > https://gitlab.com/buildroot.org/buildroot/-/jobs/6833308175 (tests.package.test_mender.TestMender) > > Runtime tested: > https://gitlab.com/kubu93/buildroot/-/jobs/6860670292 > > Signed-off-by: Romain Naour <romain.naour@smile.fr> > Cc: Christian Stewart <christian@aperture.us> Applied to master, thanks. (You had a duplicated go-bootstrap-stage3 in the title, so I removed it). Regards, Yann E. MORIN. > --- > ...ldvcs-false-when-building-go-bootstr.patch | 71 +++++++++++++++++++ > 1 file changed, 71 insertions(+) > create mode 100644 package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch > > diff --git a/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch b/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch > new file mode 100644 > index 0000000000..58f4581b02 > --- /dev/null > +++ b/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch > @@ -0,0 +1,71 @@ > +From 6b05378097c6a386ed9912d2471976dc39504e86 Mon Sep 17 00:00:00 2001 > +From: Christian Stewart <christian@aperture.us> > +Date: Thu, 27 Jul 2023 21:28:47 -0700 > +Subject: [PATCH] cmd/dist: set buildvcs=false when building go-bootstrap > + > +When building go-bootstrap as part of the make.bash process, the cmd/dist > +invokes the bootstrap Go compiler to build the go_bootstrap tool: > + > +${GOROOT_BOOTSTRAP}/bin/go install -tags=math_big_pure_go compiler_bootstrap purego bootstrap/cmd/... > + > +If there is an invalid .git directory in a parent of ${GOROOT_BOOTSTRAP}, > +make.bash will fail. Reproduction of the issue: > + > + mkdir go-issue-61620 > + cd ./go-issue-61620 > + wget https://go.dev/dl/go1.19.11.src.tar.gz > + mkdir go-bootstrap > + tar -xf go1.19.11.src.tar.gz -C ./go-bootstrap --strip-components=1 > + cd ./go-bootstrap/src/ > + bash make.bash > + cd ../../ > + wget https://go.dev/dl/go1.20.6.src.tar.gz > + mkdir go > + tar -xf go1.20.6.src.tar.gz -C ./go/ --strip-components=1 > + printf "gitdir: ../../does/not/exist/.git" > ./.git > + cd ./go/src/ > + GOROOT_BOOTSTRAP=$(pwd)/../../go-bootstrap/ bash make.bash > + > +The build fails with the following error: > + > + Building Go toolchain1 using [snip]/go-1.19.10. > + error obtaining VCS status: exit status 128 > + Use -buildvcs=false to disable VCS stamping. > + go tool dist: FAILED: [snip]/go-1.19.10/bin/go install -tags=math_big_pure_go \ > + compiler_bootstrap purego bootstrap/cmd/...: exit status 1 > + > +This change unconditionally sets -buildvcs=false when compiling go-bootstrap. We > +don't need the revision information in those binaries anyway. Setting this flag > +was previously not done as we were unsure if the go-bootstrap compiler would be > +new enough to support the buildvcs build flag. Since Go 1.20.x, Go 1.19.x is the > +minimum version for go-bootstrap, and supports -buildvcs=false. We can now set > +-buildvcs=false without worrying about compatibility. > + > +Related: https://github.com/golang/go/issues/54852 > +Fixes: https://github.com/golang/go/issues/61620 > + > +Upstream: https://github.com/golang/go/pull/61621 > + > +Signed-off-by: Christian Stewart <christian@aperture.us> > +Signed-off-by: Romain Naour <romain.naour@smile.fr> > +--- > + src/cmd/dist/buildtool.go | 3 +++ > + 1 file changed, 3 insertions(+) > + > +diff --git a/src/cmd/dist/buildtool.go b/src/cmd/dist/buildtool.go > +index a528d7aa76..3b411d6ebb 100644 > +--- a/src/cmd/dist/buildtool.go > ++++ b/src/cmd/dist/buildtool.go > +@@ -221,6 +221,9 @@ func bootstrapBuildTools() { > + cmd := []string{ > + pathf("%s/bin/go", goroot_bootstrap), > + "install", > ++ // Fixes cases where an invalid .git is present in a parent of GOROOT_BOOTSTRAP. > ++ // See: https://github.com/golang/go/issues/61620 > ++ "-buildvcs=false", > + "-tags=math_big_pure_go compiler_bootstrap purego", > + } > + if vflag > 0 { > +-- > +2.41.0 > + > -- > 2.45.0 > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
diff --git a/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch b/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch new file mode 100644 index 0000000000..58f4581b02 --- /dev/null +++ b/package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch @@ -0,0 +1,71 @@ +From 6b05378097c6a386ed9912d2471976dc39504e86 Mon Sep 17 00:00:00 2001 +From: Christian Stewart <christian@aperture.us> +Date: Thu, 27 Jul 2023 21:28:47 -0700 +Subject: [PATCH] cmd/dist: set buildvcs=false when building go-bootstrap + +When building go-bootstrap as part of the make.bash process, the cmd/dist +invokes the bootstrap Go compiler to build the go_bootstrap tool: + +${GOROOT_BOOTSTRAP}/bin/go install -tags=math_big_pure_go compiler_bootstrap purego bootstrap/cmd/... + +If there is an invalid .git directory in a parent of ${GOROOT_BOOTSTRAP}, +make.bash will fail. Reproduction of the issue: + + mkdir go-issue-61620 + cd ./go-issue-61620 + wget https://go.dev/dl/go1.19.11.src.tar.gz + mkdir go-bootstrap + tar -xf go1.19.11.src.tar.gz -C ./go-bootstrap --strip-components=1 + cd ./go-bootstrap/src/ + bash make.bash + cd ../../ + wget https://go.dev/dl/go1.20.6.src.tar.gz + mkdir go + tar -xf go1.20.6.src.tar.gz -C ./go/ --strip-components=1 + printf "gitdir: ../../does/not/exist/.git" > ./.git + cd ./go/src/ + GOROOT_BOOTSTRAP=$(pwd)/../../go-bootstrap/ bash make.bash + +The build fails with the following error: + + Building Go toolchain1 using [snip]/go-1.19.10. + error obtaining VCS status: exit status 128 + Use -buildvcs=false to disable VCS stamping. + go tool dist: FAILED: [snip]/go-1.19.10/bin/go install -tags=math_big_pure_go \ + compiler_bootstrap purego bootstrap/cmd/...: exit status 1 + +This change unconditionally sets -buildvcs=false when compiling go-bootstrap. We +don't need the revision information in those binaries anyway. Setting this flag +was previously not done as we were unsure if the go-bootstrap compiler would be +new enough to support the buildvcs build flag. Since Go 1.20.x, Go 1.19.x is the +minimum version for go-bootstrap, and supports -buildvcs=false. We can now set +-buildvcs=false without worrying about compatibility. + +Related: https://github.com/golang/go/issues/54852 +Fixes: https://github.com/golang/go/issues/61620 + +Upstream: https://github.com/golang/go/pull/61621 + +Signed-off-by: Christian Stewart <christian@aperture.us> +Signed-off-by: Romain Naour <romain.naour@smile.fr> +--- + src/cmd/dist/buildtool.go | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/src/cmd/dist/buildtool.go b/src/cmd/dist/buildtool.go +index a528d7aa76..3b411d6ebb 100644 +--- a/src/cmd/dist/buildtool.go ++++ b/src/cmd/dist/buildtool.go +@@ -221,6 +221,9 @@ func bootstrapBuildTools() { + cmd := []string{ + pathf("%s/bin/go", goroot_bootstrap), + "install", ++ // Fixes cases where an invalid .git is present in a parent of GOROOT_BOOTSTRAP. ++ // See: https://github.com/golang/go/issues/61620 ++ "-buildvcs=false", + "-tags=math_big_pure_go compiler_bootstrap purego", + } + if vflag > 0 { +-- +2.41.0 +
Since go-bootstrap-stage3 package has been added to Buildroot [1], all tests using go are failing only on Gitlab-ci jobs. The build succeed if we use docker-run script. Since the issue is only trigged by Gitlab-ci environment, the issue is related to the ownership of the (buildroot) git tree, see [2]. go seems to ignore the .gitconfig set by before_script. Anyway, go should not try to use git to retrieve some info since we are not building it from the go repository. go package was already patched for a similar issue when building host-go within docker [3]. Apply the same patch to go-bootstrap-stage3 to disable VCS support. See http://lists.busybox.net/pipermail/buildroot/2024-May/691273.html [1] f00eb37de9b4b0ddbdeb5c7ebae48c883c27e132 [2] a016b693f7830f3c8ae815851d3204b8b6e99821 [3] bc8e70a08ba083d978c828e31442a7645c9099d7 Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/6833307649 (tests.fs.test_oci.TestOci) https://gitlab.com/buildroot.org/buildroot/-/jobs/6833307823 (tests.package.test_docker_compose.TestDockerCompose) https://gitlab.com/buildroot.org/buildroot/-/jobs/6833308175 (tests.package.test_mender.TestMender) Runtime tested: https://gitlab.com/kubu93/buildroot/-/jobs/6860670292 Signed-off-by: Romain Naour <romain.naour@smile.fr> Cc: Christian Stewart <christian@aperture.us> --- ...ldvcs-false-when-building-go-bootstr.patch | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 package/go-bootstrap-stage3/0001-cmd-dist-set-buildvcs-false-when-building-go-bootstr.patch