libgo/test: Add flags to find libgcc_s in build-tree testing
diff mbox series

Message ID alpine.LFD.2.21.1911101954270.13542@redsun52.ssa.fujisawa.hgst.com
State Superseded
Headers show
Series
  • libgo/test: Add flags to find libgcc_s in build-tree testing
Related show

Commit Message

Maciej W. Rozycki Nov. 11, 2019, 3:42 p.m. UTC
Add a setting for the dynamic loader to find the shared libgcc_s library 
in build-tree testing, fixing a catastrophic libgo testsuite failure in 
cross-compilation where the library cannot be found by the loader at run 
time and consequently no test case executes, producing output (here with 
the `x86_64-linux-gnu' host and the `riscv64-linux-gnu' target, with 
RISC-V QEMU in the Linux user emulation mode as the target board) like:

spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe
./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
FAIL: archive/tar

Use the `ld_library_path' TCL variable to propagate the setting obtained 
from where GCC finds the library at link time, making test output look 
like:

spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:..././gcc/lib64/lp64d:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe
PASS
PASS: archive/tar

No summary comparison, because the libgo testsuite does not provide one 
in this configuration for some reason, however this change improves 
overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs.

	gcc/testsuite/
	* lib/go.exp (go_link_flags): Add `ld_library_path' setting to 
	find shared `libgcc_s' library at run time in build-tree 
	testing.
---
Hi,

 Regression-tested with `make check-go' with the `x86_64-linux-gnu' native 
system as well as the `x86_64-linux-gnu' host and the `riscv64-linux-gnu' 
target, with RISC-V QEMU in the Linux user emulation mode as the target 
board.

 NB as a heads-up numerous tests fail quietly (i.e. with no FAIL report 
and no name of the test case given either) to link due to unsatisfied 
symbol references, such as:

.../bin/riscv64-linux-gnu-ld: _gotest_.o: in function `cmd..z2fgo..z2finternal..z2fcache.Cache.get':
.../riscv64-linux-gnu/libgo/gotest24771/test/cache.go:182: undefined reference to `cmd..z2fgo..z2finternal..z2frenameio.ReadFile'

which I take is due to a reference to `libgotool.a' -- which is where the 
required symbols are defined -- missing from the linker invocation.  I 
don't know what's supposed to supply the library to the linker or whether 
this indeed the actual cause; I find the way libgo tests have been wired 
unusual and consequently hard to follow, so maybe someone more familiar 
with this stuff will be able to tell what is going on here.  I'll be happy 
to push any patches through testing.

 OK to apply to GCC?

  Maciej
---
 gcc/testsuite/lib/go.exp |    8 ++++++++
 1 file changed, 8 insertions(+)

gcc-test-libgo-libgcc-lib-path.diff

Comments

Ian Lance Taylor Nov. 11, 2019, 11:42 p.m. UTC | #1
[ moving from golang-dev to gofrontend-dev ]

On Mon, Nov 11, 2019 at 7:48 AM Maciej W. Rozycki <macro@wdc.com> wrote:
>
> Add a setting for the dynamic loader to find the shared libgcc_s library
> in build-tree testing, fixing a catastrophic libgo testsuite failure in
> cross-compilation where the library cannot be found by the loader at run
> time and consequently no test case executes, producing output (here with
> the `x86_64-linux-gnu' host and the `riscv64-linux-gnu' target, with
> RISC-V QEMU in the Linux user emulation mode as the target board) like:
>
> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe
> ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
> FAIL: archive/tar
>
> Use the `ld_library_path' TCL variable to propagate the setting obtained
> from where GCC finds the library at link time, making test output look
> like:
>
> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:..././gcc/lib64/lp64d:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe
> PASS
> PASS: archive/tar
>
> No summary comparison, because the libgo testsuite does not provide one
> in this configuration for some reason, however this change improves
> overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs.
>
>         gcc/testsuite/
>         * lib/go.exp (go_link_flags): Add `ld_library_path' setting to
>         find shared `libgcc_s' library at run time in build-tree
>         testing.

Is there similar code for other languages, such as Fortran?  I don't
see why Go would be different here.



>  Regression-tested with `make check-go' with the `x86_64-linux-gnu' native
> system as well as the `x86_64-linux-gnu' host and the `riscv64-linux-gnu'
> target, with RISC-V QEMU in the Linux user emulation mode as the target
> board.
>
>  NB as a heads-up numerous tests fail quietly (i.e. with no FAIL report
> and no name of the test case given either) to link due to unsatisfied
> symbol references, such as:
>
> .../bin/riscv64-linux-gnu-ld: _gotest_.o: in function `cmd..z2fgo..z2finternal..z2fcache.Cache.get':
> .../riscv64-linux-gnu/libgo/gotest24771/test/cache.go:182: undefined reference to `cmd..z2fgo..z2finternal..z2frenameio.ReadFile'
>
> which I take is due to a reference to `libgotool.a' -- which is where the
> required symbols are defined -- missing from the linker invocation.  I
> don't know what's supposed to supply the library to the linker or whether
> this indeed the actual cause; I find the way libgo tests have been wired
> unusual and consequently hard to follow, so maybe someone more familiar
> with this stuff will be able to tell what is going on here.  I'll be happy
> to push any patches through testing.

(That is, of course, a libgo test failure, and as such is not affected
by your patch to go.exp.)

In normal usage, that test is linked against libgotool.a because of
the variable extra_check_libs_cmd_go_internal_cache in
libgo/Makefile.am.  That variable is added to GOLIBS in the CHECK
variable in libgo/Makefile.am.  Maybe the fix is for
libgo/testsuite/lib/libgo.exp to use GOLIBS.

Ian
Maciej W. Rozycki Nov. 13, 2019, 1:08 a.m. UTC | #2
On Mon, 11 Nov 2019, Ian Lance Taylor wrote:

> >         gcc/testsuite/
> >         * lib/go.exp (go_link_flags): Add `ld_library_path' setting to
> >         find shared `libgcc_s' library at run time in build-tree
> >         testing.
> 
> Is there similar code for other languages, such as Fortran?  I don't
> see why Go would be different here.

 An example simulator invocation in Fortran testing here looks like:

spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:/scratch/macro/riscv-linux/obj/gcc/riscv64-linux-gnu/lib64/lp64d/libgfortran/.libs:/scratch/macro/riscv-linux/obj/gcc/riscv64-linux-gnu/lib64/lp64d/libgfortran/.libs:/scratch/macro/riscv-linux/obj/gcc/riscv64-linux-gnu/lib64/lp64d/libatomic/.libs:/scratch/macro/riscv-linux/obj/gcc/gcc:/scratch/macro/riscv-linux/obj/gcc/gcc/lib32/ilp32:/scratch/macro/riscv-linux/obj/gcc/gcc/lib32/ilp32d:/scratch/macro/riscv-linux/obj/gcc/gcc/lib64/lp64:/scratch/macro/riscv-linux/obj/gcc/gcc/lib64/lp64d:/scratch/macro/riscv-linux/obj/gcc/gcc:/scratch/macro/riscv-linux/obj/gcc/gcc/lib32/ilp32:/scratch/macro/riscv-linux/obj/gcc/gcc/lib32/ilp32d:/scratch/macro/riscv-linux/obj/gcc/gcc/lib64/lp64:/scratch/macro/riscv-linux/obj/gcc/gcc/lib64/lp64d ./alloc_comp_4.exe

and there are indeed copies of newly-built `libgcc_s' available in these 
directories, however Fortran testing is different as it is done as a part 
of GCC proper (similarly to `check-gcc-go') rather than a top-level 
library (libgfortran/ has no separate testsuite associated and 
`check-target-libgfortran' does nothing).

 I believe this is arranged by `gcc-set-multilib-library-path' in 
gcc/testsuite/lib/gcc-defs.exp, and this is does get invoked from 
`go_link_flags' too, however it has no chance to work, as the paths are 
only set if the `rootme' TCL variable is.  And `rootme' is only set via 
site.exp in gcc/ AFAICT, so all the top-level lib*/ test suites that call 
`gcc-set-multilib-library-path' are busted (unless invoked standalone with 
handcrafted site.exp or suchlike), though I guess no other one relies on 
`libgcc_s', not at least throughout, which I find plausible and which is 
why they appear to work just fine in my test environment (though I have to 
admit I haven't gone through all the testsuite failures yet, so maybe 
there is indeed a case there or a dozen that is broken).

 So I think you are right with libgo/ testing being no particularly 
different from other top-level libraries and this requires a better 
clean-up, which would go into gcc/testsuite/lib/gcc-defs.exp instead.

 In particular I think the use of `exec' is unsafe as it has a 
ridiculously short executable path length limit imposed (which I have 
actually overrun in the past in my previous test environments) and does 
not work for a remote host (as already lossily guarded against).  Both 
issues are addressed with the use of `remote_exec host', as I did with my 
proposed code.

 Also do we need to add non-selected multilib run-time load paths?  That 
seems to be the reason of the requirement to know `rootme' on one hand and 
of questionable use on the other, however the history of the change that 
introduced it is too complicated for me to know the answer the question 
offhand.  I'll have to spend some time looking into it and reading through 
past discussions, though it may take a couple days as I have an unrelated 
change outstanding that is not a bug fix and which I therefore want to 
give priority and submit before stage 1 ends.

 Please consider this patch withdrawn then, and I'll propose a replacement 
change for gcc/testsuite/lib/gcc-defs.exp soon.  We can discuss the 
concerns there.

> >  NB as a heads-up numerous tests fail quietly (i.e. with no FAIL report
> > and no name of the test case given either) to link due to unsatisfied
> > symbol references, such as:
> >
> > .../bin/riscv64-linux-gnu-ld: _gotest_.o: in function `cmd..z2fgo..z2finternal..z2fcache.Cache.get':
> > .../riscv64-linux-gnu/libgo/gotest24771/test/cache.go:182: undefined reference to `cmd..z2fgo..z2finternal..z2frenameio.ReadFile'
> >
> > which I take is due to a reference to `libgotool.a' -- which is where the
> > required symbols are defined -- missing from the linker invocation.  I
> > don't know what's supposed to supply the library to the linker or whether
> > this indeed the actual cause; I find the way libgo tests have been wired
> > unusual and consequently hard to follow, so maybe someone more familiar
> > with this stuff will be able to tell what is going on here.  I'll be happy
> > to push any patches through testing.
> 
> (That is, of course, a libgo test failure, and as such is not affected
> by your patch to go.exp.)
> 
> In normal usage, that test is linked against libgotool.a because of
> the variable extra_check_libs_cmd_go_internal_cache in
> libgo/Makefile.am.  That variable is added to GOLIBS in the CHECK
> variable in libgo/Makefile.am.  Maybe the fix is for
> libgo/testsuite/lib/libgo.exp to use GOLIBS.

 Oh, I can see what is going on here now, thanks for the hint!  That would 
be libgo/testsuite/libgo.testmain/testmain.exp though, which is where the 
compiler is called, rather than libgo/testsuite/lib/libgo.exp.  Patch sent 
separately.

  Maciej

Patch
diff mbox series

Index: gcc/gcc/testsuite/lib/go.exp
===================================================================
--- gcc.orig/gcc/testsuite/lib/go.exp
+++ gcc/gcc/testsuite/lib/go.exp
@@ -104,6 +104,14 @@  proc go_link_flags { paths } {
     verbose "shared lib extension: $shlib_ext"
 
     if { $gccpath != "" } {
+      set compiler [lindex $GOC_UNDER_TEST 0]
+      set compiler_opts [join [lrange $GOC_UNDER_TEST 1 end]]
+      set libgccs [remote_exec host "$compiler" \
+		   "$compiler_opts -print-file-name=libgcc_s.${shlib_ext}"]
+      if { [lindex $libgccs 0] == 0 \
+	   && [set libgccsdir [file dirname [lindex $libgccs 1]]] != "" } {
+          append ld_library_path ":${libgccsdir}"
+      }
       if [file exists "${gccpath}/libgo/libgobegin.a"] {
 	  append flags "-L${gccpath}/libgo "
       }