| Message ID | alpine.LFD.2.21.1911290526340.13542@redsun52.ssa.fujisawa.hgst.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | Fix run-time handling of `libgcc_s' in testing | expand |
On Fri, Nov 29, 2019 at 1:09 AM Maciej W. Rozycki <macro@wdc.com> wrote: > > Fix a catastrophic libgo testsuite failure in cross-compilation where > the shared `libgcc_s' library cannot be found by the loader at run time > in build-tree testing and consequently all test cases fail the execution > stage, giving output (here with the `x86_64-linux-gnu' host and the > `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation > mode as the target board) like: > > spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe > ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory > FAIL: archive/tar > > To do so rework `gcc-set-multilib-library-path' so as not to rely on the > `rootme' TCL variable to have been preset in testsuite invocation, which > only works for the GCC test suites and not for library test suites, and > also use `remote_exec host' rather than `exec' to invoke the compiler in > determination of `libgcc_s' locations, so that the solution works in > remote testing as well while also avoiding the hardcoded limit of the > executable's path length imposed by `exec'. > > This is based on an observation that the multilib root directory can be > determined by stripping out the multilib directory in effect as printed > with the `-print-multi-directory' option from the path produced by the > `-print-file-name=' option. And then individual full multilib paths can > be assembled for the other multilibs by appending their respective > multilib directories to the multilib root directory. > > Unlike with the old solution the full multilib paths are not checked for > the presence of the shared `libgcc_s' library there, but that is > supposed to be harmless. Also the full multilib path for the multilib > used with the compiler used for testing will now come first, which > should reduce run-time processing in the usual case. > > With this change in place test output instead looks like: > > spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs:..././gcc/lib64/lp64d:..././gcc/.:..././gcc/lib32/ilp32:..././gcc/lib32/ilp32d:..././gcc/lib64/lp64 ./a.exe > PASS > PASS: archive/tar > > No summary comparison, because the libgo testsuite does not provide one > in this configuration for some reason, however this change improves > overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs. > > gcc/testsuite/ > * lib/gcc-defs.exp (gcc-set-multilib-library-path): Use > `-print-file-name=' to determine the multilib root directory. > Use `remote_exec host' rather than `exec' to invoke the > compiler. This patch is for better Go support but the patch is to generic code. The patch is fine with me but should ideally be reviewed by a testsuite maintainer. Ian
On Mon, 2 Dec 2019, Ian Lance Taylor wrote: > > gcc/testsuite/ > > * lib/gcc-defs.exp (gcc-set-multilib-library-path): Use > > `-print-file-name=' to determine the multilib root directory. > > Use `remote_exec host' rather than `exec' to invoke the > > compiler. > > This patch is for better Go support but the patch is to generic code. > The patch is fine with me but should ideally be reviewed by a > testsuite maintainer. I agree. I have cc-ed you to keep you in the circle and I have added the testsuite people now too, but ultimately I think it's Richard who might be the best person to look into it as he's been highly skilled in this area and he has also dealt with this particular piece in the past. Richard, would you be able to find a few cycles and be kind enough to look into it? I'm somewhat surprised it's only libgo that suffers from the lack of run-time library paths in my setup where a random copy of `libgcc_s' is not already present in the environment, but maybe no other target library test suite pulls `libgcc_s' with any of its tests. Maciej
On Fri, 29 Nov 2019, Maciej W. Rozycki wrote: > Fix a catastrophic libgo testsuite failure in cross-compilation where > the shared `libgcc_s' library cannot be found by the loader at run time > in build-tree testing and consequently all test cases fail the execution > stage, giving output (here with the `x86_64-linux-gnu' host and the > `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation > mode as the target board) like: > > spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe > ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory > FAIL: archive/tar Ping for: <https://gcc.gnu.org/ml/gcc-patches/2019-11/msg02610.html> Maciej
On Fri, 29 Nov 2019, Maciej W. Rozycki wrote: > Fix a catastrophic libgo testsuite failure in cross-compilation where > the shared `libgcc_s' library cannot be found by the loader at run time > in build-tree testing and consequently all test cases fail the execution > stage, giving output (here with the `x86_64-linux-gnu' host and the > `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation > mode as the target board) like: > > spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe > ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory > FAIL: archive/tar Ping for: <https://gcc.gnu.org/ml/gcc-patches/2019-11/msg02610.html> Maciej
On Dec 9, 2019, at 1:30 PM, Maciej W. Rozycki <macro@wdc.com> wrote: > > On Fri, 29 Nov 2019, Maciej W. Rozycki wrote: > >> Fix a catastrophic libgo testsuite failure in cross-compilation where >> the shared `libgcc_s' library cannot be found by the loader at run time >> in build-tree testing and consequently all test cases fail the execution >> stage, giving output (here with the `x86_64-linux-gnu' host and the >> `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation >> mode as the target board) like: >> >> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe >> ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory >> FAIL: archive/tar > > Ping for: > > <https://gcc.gnu.org/ml/gcc-patches/2019-11/msg02610.html> Ok. Watch for any breakage.
On Fri, 20 Dec 2019, Mike Stump wrote: > >> Fix a catastrophic libgo testsuite failure in cross-compilation where > >> the shared `libgcc_s' library cannot be found by the loader at run time > >> in build-tree testing and consequently all test cases fail the execution > >> stage, giving output (here with the `x86_64-linux-gnu' host and the > >> `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation > >> mode as the target board) like: > >> > >> spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe > >> ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory > >> FAIL: archive/tar > > > > Ping for: > > > > <https://gcc.gnu.org/ml/gcc-patches/2019-11/msg02610.html> > > Ok. Applied now, thanks for your review. > Watch for any breakage. Will do. Maciej
Index: gcc/gcc/testsuite/lib/gcc-defs.exp =================================================================== --- gcc.orig/gcc/testsuite/lib/gcc-defs.exp +++ gcc/gcc/testsuite/lib/gcc-defs.exp @@ -324,29 +324,44 @@ proc dg-additional-files-options { optio # for COMPILER, including multilib directories. proc gcc-set-multilib-library-path { compiler } { - global rootme + set shlib_ext [get_shlib_extension] + set options [lrange $compiler 1 end] + set compiler [lindex $compiler 0] - # ??? rootme will not be set when testing an installed compiler. - # In that case, we should perhaps use some other method to find - # libraries. - if {![info exists rootme]} { + set libgcc_s_x [remote_exec host "$compiler" \ + "$options -print-file-name=libgcc_s.${shlib_ext}"] + if { [lindex $libgcc_s_x 0] == 0 \ + && [set libgcc_s_dir [file dirname [lindex $libgcc_s_x 1]]] != "" } { + set libpath ":${libgcc_s_dir}" + } else { return "" } - set libpath ":${rootme}" - set options [lrange $compiler 1 end] - set compiler [lindex $compiler 0] - if { [is_remote host] == 0 && [which $compiler] != 0 } { - foreach i "[eval exec $compiler $options --print-multi-lib]" { + set multi_dir_x [remote_exec host "$compiler" \ + "$options -print-multi-directory"] + set multi_lib_x [remote_exec host "$compiler" \ + "$options -print-multi-lib"] + if { [lindex $multi_dir_x 0] == 0 && [lindex $multi_lib_x 0] == 0 } { + set multi_dir [string trim [lindex $multi_dir_x 1]] + set multi_lib [string trim [lindex $multi_lib_x 1]] + if { "$multi_dir" == "." } { + set multi_root "$libgcc_s_dir" + } else { + set multi_match [string last "/$multi_dir" "$libgcc_s_dir"] + if { "$multi_match" < 0 } { + return $libpath + } + set multi_root [string range "$libgcc_s_dir" \ + 0 [expr $multi_match - 1]] + } + foreach i "$multi_lib" { set mldir "" regexp -- "\[a-z0-9=_/\.-\]*;" $i mldir set mldir [string trimright $mldir "\;@"] - if { "$mldir" == "." } { + if { "$mldir" == "$multi_dir" } { continue } - if { [llength [glob -nocomplain ${rootme}/${mldir}/libgcc_s*.so.*]] >= 1 } { - append libpath ":${rootme}/${mldir}" - } + append libpath ":${multi_root}/${mldir}" } }
Fix a catastrophic libgo testsuite failure in cross-compilation where the shared `libgcc_s' library cannot be found by the loader at run time in build-tree testing and consequently all test cases fail the execution stage, giving output (here with the `x86_64-linux-gnu' host and the `riscv64-linux-gnu' target, with RISC-V QEMU in the Linux user emulation mode as the target board) like: spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs ./a.exe ./a.exe: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory FAIL: archive/tar To do so rework `gcc-set-multilib-library-path' so as not to rely on the `rootme' TCL variable to have been preset in testsuite invocation, which only works for the GCC test suites and not for library test suites, and also use `remote_exec host' rather than `exec' to invoke the compiler in determination of `libgcc_s' locations, so that the solution works in remote testing as well while also avoiding the hardcoded limit of the executable's path length imposed by `exec'. This is based on an observation that the multilib root directory can be determined by stripping out the multilib directory in effect as printed with the `-print-multi-directory' option from the path produced by the `-print-file-name=' option. And then individual full multilib paths can be assembled for the other multilibs by appending their respective multilib directories to the multilib root directory. Unlike with the old solution the full multilib paths are not checked for the presence of the shared `libgcc_s' library there, but that is supposed to be harmless. Also the full multilib path for the multilib used with the compiler used for testing will now come first, which should reduce run-time processing in the usual case. With this change in place test output instead looks like: spawn qemu-riscv64 -E LD_LIBRARY_PATH=.:.../riscv64-linux-gnu/lib64/lp64d/libgo/.libs:..././gcc/lib64/lp64d:..././gcc/.:..././gcc/lib32/ilp32:..././gcc/lib32/ilp32d:..././gcc/lib64/lp64 ./a.exe PASS PASS: archive/tar No summary comparison, because the libgo testsuite does not provide one in this configuration for some reason, however this change improves overall results from 0 PASSes and 159 FAILs to 133 PASSes and 26 FAILs. gcc/testsuite/ * lib/gcc-defs.exp (gcc-set-multilib-library-path): Use `-print-file-name=' to determine the multilib root directory. Use `remote_exec host' rather than `exec' to invoke the compiler. --- Hi, As PR testsuite/40699, PR testsuite/40707 and PR testsuite/40709 and resulting r149508, a revert of r149113 ("Tidy up testsuite handling of LD_LIBRARY_PATH"), <https://gcc.gnu.org/ml/gcc-patches/2009-06/msg00151.html> and some other commits, have indicated non-selected multilib directories have to be included in the dynamic loader's library path for some targets for some reason. So I have decided to preserve the approach, even though it appears odd to me. Maciej Changes from v1: - Resolve the issue globally in `gcc-set-multilib-library-path' in gcc-defs.exp rather than for libgo only in `go_link_flags' in go.exp. --- gcc/testsuite/lib/gcc-defs.exp | 43 +++++++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 14 deletions(-)