[gotools] Fix some gotools testing problems

Message ID yddo9qgrr6t.fsf@CeBiTec.Uni-Bielefeld.DE
State New
Headers show
Series
  • [gotools] Fix some gotools testing problems
Related show

Commit Message

Rainer Orth Sept. 12, 2017, 1:18 p.m.
A couple of gotools test FAIL on Solaris, and there are several issues
that make investigation particularly tedious.

* The one invocation of gotest doesn't support passing additional flags
  (--keep in particular).  Added below.

* The order of some of the summaries isn't stable (cmd/go and runtime,
  while cgo and carchive are, it seems).  I'm now sorting the output by
  test name, both for make check output and the creation of gotools.sum.

  Otherwise, it's hard to compare mail-report.log between builds.

* The freshly built go, cgo, and gofmt rely on LD_LIBRARY_PATH to be set
  so libgcc.so is found at runtime.  The right value is passed in from
  the toplevel during a bootstrap, but missing when I run make check in
  gotools manually.

* The most glaring problem is getting your hands on the failing
  executables: there are several levels of indirection here

  make -> gotest ... -> go build -> gccgo -> a.out
       -> go test
       
  Many of those steps also depend on lots of environment variables set
  to the check-gcc or check-gccgo wrapper scripts and it's hard to
  determine what's necessary to be able to lay my hand on the resulting
  executable to run it under gdb.  I've found no way to pass e.g. -work
  to go build from the outside (e.g. via an environment variable), nor
  can I see the commands executed, so this means modifying the test
  sources or run the full testsuite under truss in the hope to determine
  all that's necessary to reproduce the build.

Anyway, here's the trivial part so far.  Ok for mainline?

	Rainer

Comments

Ian Lance Taylor via gcc-patches Sept. 18, 2017, 9:24 p.m. | #1
On Tue, Sep 12, 2017 at 6:18 AM, Rainer Orth
<ro@cebitec.uni-bielefeld.de> wrote:
>
> A couple of gotools test FAIL on Solaris, and there are several issues
> that make investigation particularly tedious.
>
> * The one invocation of gotest doesn't support passing additional flags
>   (--keep in particular).  Added below.
>
> * The order of some of the summaries isn't stable (cmd/go and runtime,
>   while cgo and carchive are, it seems).  I'm now sorting the output by
>   test name, both for make check output and the creation of gotools.sum.
>
>   Otherwise, it's hard to compare mail-report.log between builds.
>
> * The freshly built go, cgo, and gofmt rely on LD_LIBRARY_PATH to be set
>   so libgcc.so is found at runtime.  The right value is passed in from
>   the toplevel during a bootstrap, but missing when I run make check in
>   gotools manually.
>
> * The most glaring problem is getting your hands on the failing
>   executables: there are several levels of indirection here
>
>   make -> gotest ... -> go build -> gccgo -> a.out
>        -> go test
>
>   Many of those steps also depend on lots of environment variables set
>   to the check-gcc or check-gccgo wrapper scripts and it's hard to
>   determine what's necessary to be able to lay my hand on the resulting
>   executable to run it under gdb.  I've found no way to pass e.g. -work
>   to go build from the outside (e.g. via an environment variable), nor
>   can I see the commands executed, so this means modifying the test
>   sources or run the full testsuite under truss in the hope to determine
>   all that's necessary to reproduce the build.
>
> Anyway, here's the trivial part so far.  Ok for mainline?
>
>         Rainer
>
> --
> -----------------------------------------------------------------------------
> Rainer Orth, Center for Biotechnology, Bielefeld University
>
>
> 2017-09-12  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>
>
>         * Makefile.am (GOTESTFLAGS): New variable.
>         (check-runtime): Pass it to gotest.
>         (check-go-tools): Sort summary.
>         (check-runtime): Likewise.
>         (check-cgo-test): Likewise.
>         (check-carchive-test): Likewise.
>         (check): Likewise.
>         * Makefile.in: Regenerate.


This patch is OK.  Go ahead and commit.  Thanks.

What I do for debugging is to run `make install`.  Then I can simply
run `GO_TESTING_GOTOOLS=yes go test` in the appropriate directory.  I
don't have a good solution for simpler debugging without running `make
install` first.

Ian

Patch

# HG changeset patch
# Parent  6cf2b85bf501a5d54f0e0558b078b90b6d6ea0dc
Fix some gotools testing problems

diff --git a/gotools/Makefile.am b/gotools/Makefile.am
--- a/gotools/Makefile.am
+++ b/gotools/Makefile.am
@@ -139,6 +139,8 @@  install-exec-local: cgo$(EXEEXT)
 uninstall-local:
 	rm -f $(DESTDIR)$(libexecsubdir)/cgo$(exeext)
 
+GOTESTFLAGS =
+
 # Run tests using the go tool, and frob the output to look like that
 # generated by DejaGNU.  The main output of this is two files:
 # gotools.sum and gotools.log.
@@ -211,7 +213,7 @@  check-go-tool: go$(EXEEXT) cgo$(EXEEXT) 
 	GOPATH=`cd check-go-dir && $(PWD_COMMAND)`; \
 	export GOPATH; \
 	(cd check-go-dir/src/cmd/go && $(abs_builddir)/go$(EXEEXT) test -test.short -test.v) > cmd_go-testlog 2>&1 || true
-	grep '^--- ' cmd_go-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/'
+	grep '^--- ' cmd_go-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' | sort -k 2
 
 # check-runtime runs `go test runtime` in our environment.
 # The runtime package is also tested as part of libgo,
@@ -225,15 +227,15 @@  check-runtime: go$(EXEEXT) cgo$(EXEEXT) 
 	GOARCH=`$(abs_builddir)/go$(EXEEXT) env GOARCH`; \
 	GOOS=`$(abs_builddir)/go$(EXEEXT) env GOOS`; \
 	files=`$(SHELL) $(libgosrcdir)/../match.sh --goarch=$${GOARCH} --goos=$${GOOS} --srcdir=$(libgosrcdir)/runtime --extrafiles="$(libgodir)/runtime_sysinfo.go $(libgodir)/sigtab.go" --tag=libffi`; \
-	echo "$(ECHO_ENV) GC='$(abs_builddir)/check-gccgo -fgo-compiling-runtime' GOARCH=$${GOARCH} GOOS=$${GOOS} $(SHELL) $(libgosrcdir)/../testsuite/gotest --goarch=$${GOARCH} --goos=$${GOOS} --basedir=$(libgosrcdir)/.. --srcdir=$(libgosrcdir)/runtime --pkgpath=runtime --pkgfiles='$${files}' -test.v" > runtime-testlog
+	echo "$(ECHO_ENV) GC='$(abs_builddir)/check-gccgo -fgo-compiling-runtime' GOARCH=$${GOARCH} GOOS=$${GOOS} $(SHELL) $(libgosrcdir)/../testsuite/gotest --goarch=$${GOARCH} --goos=$${GOOS} --basedir=$(libgosrcdir)/.. --srcdir=$(libgosrcdir)/runtime --pkgpath=runtime --pkgfiles='$${files}' $(GOTESTFLAGS) -test.v" > runtime-testlog
 	$(CHECK_ENV) \
 	GC="$${GCCGO} -fgo-compiling-runtime"; \
 	export GC; \
 	GOARCH=`$(abs_builddir)/go$(EXEEXT) env GOARCH`; \
 	GOOS=`$(abs_builddir)/go$(EXEEXT) env GOOS`; \
 	files=`$(SHELL) $(libgosrcdir)/../match.sh --goarch=$${GOARCH} --goos=$${GOOS} --srcdir=$(libgosrcdir)/runtime --extrafiles="$(libgodir)/runtime_sysinfo.go $(libgodir)/sigtab.go" --tag=libffi`; \
-	$(SHELL) $(libgosrcdir)/../testsuite/gotest --goarch=$${GOARCH} --goos=$${GOOS} --basedir=$(libgosrcdir)/.. --srcdir=$(libgosrcdir)/runtime --pkgpath=runtime --pkgfiles="$${files}" -test.v >> runtime-testlog 2>&1 || true
-	grep '^--- ' runtime-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/'
+	$(SHELL) $(libgosrcdir)/../testsuite/gotest --goarch=$${GOARCH} --goos=$${GOOS} --basedir=$(libgosrcdir)/.. --srcdir=$(libgosrcdir)/runtime --pkgpath=runtime --pkgfiles="$${files}" $(GOTESTFLAGS) -test.v >> runtime-testlog 2>&1 || true
+	grep '^--- ' runtime-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' | sort -k 2
 
 # check-cgo-test runs `go test misc/cgo/test` in our environment.
 check-cgo-test: go$(EXEEXT) cgo$(EXEEXT) check-head check-gccgo check-gcc
@@ -246,7 +248,7 @@  check-cgo-test: go$(EXEEXT) cgo$(EXEEXT)
 	GOTRACEBACK=2; \
 	export GOTRACEBACK; \
 	(cd cgo-test-dir/misc/cgo/test && $(abs_builddir)/go$(EXEEXT) test -test.short -test.v) >> cgo-testlog 2>&1 || true
-	grep '^--- ' cgo-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/'
+	grep '^--- ' cgo-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' | sort -k 2
 
 # check-carchive-test runs `go test misc/cgo/testcarchive/carchive_test.go`
 # in our environment.
@@ -260,7 +262,7 @@  check-carchive-test: go$(EXEEXT) cgo$(EX
 	LIBRARY_PATH=`echo $${abs_libgodir}/.libs:$${LIBRARY_PATH} | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; \
 	export LIBRARY_PATH; \
 	(cd carchive-test-dir/misc/cgo/testcarchive && $(abs_builddir)/go$(EXEEXT) test -test.v carchive_test.go) >> carchive-testlog 2>&1 || true
-	grep '^--- ' carchive-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/'
+	grep '^--- ' carchive-testlog | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' | sort -k 2
 
 # The check targets runs the tests and assembles the output files.
 check: check-head check-go-tool check-runtime check-cgo-test check-carchive-test
@@ -271,7 +273,7 @@  check: check-head check-go-tool check-ru
 	  echo "Running $${testname}" >> gotools.sum; \
 	  echo "Running $${testname}" >> gotools.log; \
 	  sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' < $${file} >> gotools.log; \
-	  grep '^--- ' $${file} | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' -e 's/SKIP/UNTESTED/' >> gotools.sum; \
+	  grep '^--- ' $${file} | sed -e 's/^--- \(.*\) ([^)]*)$$/\1/' -e 's/SKIP/UNTESTED/' | sort -k 2 >> gotools.sum; \
 	done
 	@echo >> gotools.sum
 	@echo "		=== gotools Summary ===" >> gotools.sum