| Message ID | 20210105214349.3401864-1-ibuclaw@gdcproject.org |
|---|---|
| State | New |
| Headers | show |
| Series | libphobos: Allow building libphobos using Solaris/x86 assembler | expand |
Hi Iain, > This patch removes the disabling of libphobos when the Solaris/x86 > assembler is being used. > > Since r11-6373, D symbols are now compressed using back references, this > helped reduce the average symbol length by a factor of about 3, while > the longest symbol shrank from 416133 to 1142 characters. So the issues > that were seen on Solaris/x86 should no longer be a problem. > > However, I have only used x86_64-apple-darwin10 for testing, as > libphobos couldn't be built on that target for the same reason, except > it was the system linker segfaulting due to long symbol names. > > It would be good to know if Solaris has also benefitted from the change. great, thanks. I'll give this a whirl once today's regular bootstraps have finished. Rainer
Hi Iain, >> This patch removes the disabling of libphobos when the Solaris/x86 >> assembler is being used. >> >> Since r11-6373, D symbols are now compressed using back references, this >> helped reduce the average symbol length by a factor of about 3, while >> the longest symbol shrank from 416133 to 1142 characters. So the issues >> that were seen on Solaris/x86 should no longer be a problem. >> >> However, I have only used x86_64-apple-darwin10 for testing, as >> libphobos couldn't be built on that target for the same reason, except >> it was the system linker segfaulting due to long symbol names. >> >> It would be good to know if Solaris has also benefitted from the change. > > great, thanks. I'll give this a whirl once today's regular bootstraps > have finished. here's what I found: the build itself worked just fine and the libphobos test results are identical to those with gas. However, a few gdc tests fail when Solaris/x86 as is used, for two reasons: +UNRESOLVED: gdc.test/runnable/mangle.d compilation failed to produce executable +UNRESOLVED: gdc.test/runnable/mangle.d -shared-libphobos compilation failed to produce executable Assembler: mangle.d "/var/tmp//ccG72ALc.s", line 200 : Syntax error Near line: " movzbl test_эльфийские_письмена_9, %eax" [...] +UNRESOLVED: gdc.test/runnable/testmodule.d compilation failed to produce executable +UNRESOLVED: gdc.test/runnable/testmodule.d -shared-libphobos compilation failed to produce executable Assembler: testmodule.d "/var/tmp//ccw9j5oa.s", line 20 : Syntax error Near line: " call _D7dstress3run17unicode_06_哪里6哪里FiZi" [...] +UNRESOLVED: gdc.test/runnable/ufcs.d compilation failed to produce executable +UNRESOLVED: gdc.test/runnable/ufcs.d -shared-libphobos compilation failed to produce executable Assembler: ufcs.d "/var/tmp//ccWd6kud.s", line 7774 : Syntax error Near line: " .globl _D4ufcs6α8503FiZv" [...] The Solaris assemblers don't support UTF-8 identifiers. Unless gdc can encode them in some way for toolchains like this (no idea if this is worth the effort), it may be possible to guard the tests with the ucn effective-target keyword. Apart from that, it seems strange that the failing tests should only show up as UNSUPPORTED. I'd have expected the compilation to FAIL, but IIRC the gdc testsuite has to ignore all output, so the test for excess errors which would usually catch this is disabled effectively. The last failure is different and due to how COMDAT group handling is done with Solaris as: +UNRESOLVED: gdc.test/runnable/test42.d compilation failed to produce executable +UNRESOLVED: gdc.test/runnable/test42.d -shared-libphobos compilation failed to produce executable which yields Input string too long, limit 10240 The offending input lines are (stripped for brevity) .section .tdata._D6test42__T5Foo71VAyaa2623[...] .group _D6test42__T5Foo71VAyaa2623_68656c6c6f616[...] The first line is 10597 chars, the second even 15869. Rainer
Excerpts from Rainer Orth's message of January 6, 2021 2:57 pm: > Hi Iain, > >>> This patch removes the disabling of libphobos when the Solaris/x86 >>> assembler is being used. >>> >>> Since r11-6373, D symbols are now compressed using back references, this >>> helped reduce the average symbol length by a factor of about 3, while >>> the longest symbol shrank from 416133 to 1142 characters. So the issues >>> that were seen on Solaris/x86 should no longer be a problem. >>> >>> However, I have only used x86_64-apple-darwin10 for testing, as >>> libphobos couldn't be built on that target for the same reason, except >>> it was the system linker segfaulting due to long symbol names. >>> >>> It would be good to know if Solaris has also benefitted from the change. >> >> great, thanks. I'll give this a whirl once today's regular bootstraps >> have finished. > > here's what I found: the build itself worked just fine and the libphobos > test results are identical to those with gas. However, a few gdc tests > fail when Solaris/x86 as is used, for two reasons: > > +UNRESOLVED: gdc.test/runnable/mangle.d compilation failed to produce executable > +UNRESOLVED: gdc.test/runnable/mangle.d -shared-libphobos compilation failed to produce executable > > Assembler: mangle.d > "/var/tmp//ccG72ALc.s", line 200 : Syntax error > Near line: " movzbl test_эльфийские_письмена_9, %eax" > [...] > > +UNRESOLVED: gdc.test/runnable/testmodule.d compilation failed to produce executable > +UNRESOLVED: gdc.test/runnable/testmodule.d -shared-libphobos compilation failed to produce executable > > Assembler: testmodule.d > "/var/tmp//ccw9j5oa.s", line 20 : Syntax error > Near line: " call _D7dstress3run17unicode_06_哪里6哪里FiZi" > [...] > > +UNRESOLVED: gdc.test/runnable/ufcs.d compilation failed to produce executable > +UNRESOLVED: gdc.test/runnable/ufcs.d -shared-libphobos compilation failed to produce executable > > Assembler: ufcs.d > "/var/tmp//ccWd6kud.s", line 7774 : Syntax error > Near line: " .globl _D4ufcs6α8503FiZv" > [...] > > The Solaris assemblers don't support UTF-8 identifiers. Unless gdc can > encode them in some way for toolchains like this (no idea if this is > worth the effort), it may be possible to guard the tests with the ucn > effective-target keyword. > > Apart from that, it seems strange that the failing tests should only > show up as UNSUPPORTED. I'd have expected the compilation to FAIL, but > IIRC the gdc testsuite has to ignore all output, so the test for excess > errors which would usually catch this is disabled effectively. > Indeed, the testsuite is far too verbose. Although many tests have a TEST_OUTPUT directive, converting them to a Dejagnu style is probably too much effort for the gain. Those tests can just be explicitly disabled, I'll look into that. > The last failure is different and due to how COMDAT group handling is > done with Solaris as: > > +UNRESOLVED: gdc.test/runnable/test42.d compilation failed to produce executable > +UNRESOLVED: gdc.test/runnable/test42.d -shared-libphobos compilation failed to produce executable > > which yields > > Input string too long, limit 10240 > > The offending input lines are (stripped for brevity) > > .section .tdata._D6test42__T5Foo71VAyaa2623[...] > .group _D6test42__T5Foo71VAyaa2623_68656c6c6f616[...] > > The first line is 10597 chars, the second even 15869. > Is there a max symbol length macro available internally? Maybe could just compress symbols using MD5 if they exceed a certain length... Iain.
Hi Iain, >> The Solaris assemblers don't support UTF-8 identifiers. Unless gdc can >> encode them in some way for toolchains like this (no idea if this is >> worth the effort), it may be possible to guard the tests with the ucn >> effective-target keyword. >> >> Apart from that, it seems strange that the failing tests should only >> show up as UNSUPPORTED. I'd have expected the compilation to FAIL, but >> IIRC the gdc testsuite has to ignore all output, so the test for excess >> errors which would usually catch this is disabled effectively. > > Indeed, the testsuite is far too verbose. Although many tests have a > TEST_OUTPUT directive, converting them to a Dejagnu style is probably > too much effort for the gain. > > Those tests can just be explicitly disabled, I'll look into that. Great, thanks. >> The last failure is different and due to how COMDAT group handling is >> done with Solaris as: >> >> +UNRESOLVED: gdc.test/runnable/test42.d compilation failed to produce >> executable >> +UNRESOLVED: gdc.test/runnable/test42.d -shared-libphobos compilation >> failed to produce executable >> >> which yields >> >> Input string too long, limit 10240 >> >> The offending input lines are (stripped for brevity) >> >> .section .tdata._D6test42__T5Foo71VAyaa2623[...] >> .group _D6test42__T5Foo71VAyaa2623_68656c6c6f616[...] >> >> The first line is 10597 chars, the second even 15869. >> > > Is there a max symbol length macro available internally? Maybe could Not that I'm aware of. I believe D tests are the first time ever that I ran into this Solaris/x86 as limit. One might try to iteratively determine the value at configure time if this is helpful. No idea if other non-gas assemblers are even worse in that apartment. E.g. the Solaris/SPARC one has a considerably higher limit... > just compress symbols using MD5 if they exceed a certain length... That's certainly an easy option. OTOH if this is unlikely to occur in real-life code, once could just xfail the test on Solaris/x86 with as... Rainer
Excerpts from Rainer Orth's message of January 7, 2021 5:17 pm: > Hi Iain, > >>> The Solaris assemblers don't support UTF-8 identifiers. Unless gdc can >>> encode them in some way for toolchains like this (no idea if this is >>> worth the effort), it may be possible to guard the tests with the ucn >>> effective-target keyword. >>> >>> Apart from that, it seems strange that the failing tests should only >>> show up as UNSUPPORTED. I'd have expected the compilation to FAIL, but >>> IIRC the gdc testsuite has to ignore all output, so the test for excess >>> errors which would usually catch this is disabled effectively. >> >> Indeed, the testsuite is far too verbose. Although many tests have a >> TEST_OUTPUT directive, converting them to a Dejagnu style is probably >> too much effort for the gain. >> >> Those tests can just be explicitly disabled, I'll look into that. > > Great, thanks. > >>> The last failure is different and due to how COMDAT group handling is >>> done with Solaris as: >>> >>> +UNRESOLVED: gdc.test/runnable/test42.d compilation failed to produce >>> executable >>> +UNRESOLVED: gdc.test/runnable/test42.d -shared-libphobos compilation >>> failed to produce executable >>> >>> which yields >>> >>> Input string too long, limit 10240 >>> >>> The offending input lines are (stripped for brevity) >>> >>> .section .tdata._D6test42__T5Foo71VAyaa2623[...] >>> .group _D6test42__T5Foo71VAyaa2623_68656c6c6f616[...] >>> >>> The first line is 10597 chars, the second even 15869. >>> >> >> Is there a max symbol length macro available internally? Maybe could > > Not that I'm aware of. I believe D tests are the first time ever that I > ran into this Solaris/x86 as limit. One might try to iteratively > determine the value at configure time if this is helpful. No idea if > other non-gas assemblers are even worse in that apartment. E.g. the > Solaris/SPARC one has a considerably higher limit... > >> just compress symbols using MD5 if they exceed a certain length... > > That's certainly an easy option. OTOH if this is unlikely to occur in > real-life code, once could just xfail the test on Solaris/x86 with as... > Looking at the test which generates that symbol, it is very much a contrived example that deliberately exceeds the limit of another linker. However given the size and use of meta and templates in some production codebases out there, I can't say that symbols like it for sure won't appear in real-life code. Though it's going to be unlikely said code will be compiled for Solaris anyway. Iain.
Excerpts from Iain Buclaw's message of January 7, 2021 6:48 pm: > Excerpts from Rainer Orth's message of January 7, 2021 5:17 pm: >> Hi Iain, >> >>>> The Solaris assemblers don't support UTF-8 identifiers. Unless gdc can >>>> encode them in some way for toolchains like this (no idea if this is >>>> worth the effort), it may be possible to guard the tests with the ucn >>>> effective-target keyword. >>>> >>>> Apart from that, it seems strange that the failing tests should only >>>> show up as UNSUPPORTED. I'd have expected the compilation to FAIL, but >>>> IIRC the gdc testsuite has to ignore all output, so the test for excess >>>> errors which would usually catch this is disabled effectively. >>> >>> Indeed, the testsuite is far too verbose. Although many tests have a >>> TEST_OUTPUT directive, converting them to a Dejagnu style is probably >>> too much effort for the gain. >>> >>> Those tests can just be explicitly disabled, I'll look into that. >> >> Great, thanks. >> Having a look, actually I can just use the presence of TEST_OUTPUT to be a gate for whether to prune all output or not. Maybe this can be improved later to extract the contents of TEST_OUTPUT, but for now, it has caught a few hidden bugs in the tests that I have handled in the patch below (no changelog entry yet). Iain. --- diff --git a/gcc/testsuite/lib/gdc-utils.exp b/gcc/testsuite/lib/gdc-utils.exp index 6d4a15e9a67..3d9fd401ee8 100644 --- a/gcc/testsuite/lib/gdc-utils.exp +++ b/gcc/testsuite/lib/gdc-utils.exp @@ -173,6 +173,27 @@ proc gdc-copy-extra { base extra } { return $extra } +# +# Some tests in the DMD testsuite have specific target requirements, +# handle them explicitly here. +# + +proc gdc-extra-test-options { fdout test } { + switch $test { + "runnable/mangle.d" - + "runnable/testmodule.d" - + "runnable/ufcs.d" { + # Tests that require effective-target-ucn + puts $fdout "// { dg-skip-if \"\" { ! ucn } }" + } + + "runnable/test42.d" { + # Tests that overflow line limits of older assemblers. + puts $fdout "// { dg-xfail-if \"Lines exceed 10240 characters\" { *-*-solaris2.* && { ! gas } } }" + } + } +} + # # Translate DMD test directives to dejagnu equivalent. # @@ -209,8 +230,10 @@ proc gdc-convert-test { base test } { set extra_sources "" set extra_files "" + set ddoc_options "" set needs_phobos 0 set saw_test_flags 0 + set saw_test_output 0 upvar 1 compilable_do_what compilable_do_what set compilable_output_file_ext "" @@ -243,6 +266,14 @@ proc gdc-convert-test { base test } { # LINK sets dg-do-what-default "link" set compilable_do_what "link" + } elseif [regexp -- {TEST_OUTPUT} $copy_line] { + # TEST_OUTPUT contents are ignored, but it might be possible to + # convert it into a series of either dg-prune or dg-errors. + # Currently, only saw_test_output is set so that dg-prune is + # added before running the test. + regsub -- {TEST_OUTPUT.*$} $copy_line "" out_line + set saw_test_output 1 + } elseif [regexp -- {POST_SCRIPT} $copy_line] { # POST_SCRIPT is not handled regsub -- {POST_SCRIPT.*$} $copy_line "" out_line @@ -276,7 +307,15 @@ proc gdc-convert-test { base test } { } elseif [regexp -- {EXTRA_SOURCES\s*:\s*(.*)} $copy_line match sources] { # EXTRA_SOURCES are appended to extra_sources list foreach srcfile $sources { - lappend extra_sources $srcfile + # Ddoc files are not handled by the compiler directly, they are + # instead passed in using -fdoc-inc= + if [regexp -- {\.ddoc$} $srcfile extmatch] { + lappend extra_files $srcfile + lappend ddoc_options "-fdoc-inc=$type/$srcfile" + + } else { + lappend extra_sources $srcfile + } } regsub -- {EXTRA_SOURCES.*$} $copy_line "" out_line @@ -311,7 +350,7 @@ proc gdc-convert-test { base test } { } elseif [regexp -- {COMPILABLE_MATH_TEST} $copy_line match sources] { # COMPILABLE_MATH_TEST annotates tests that import the std.math # module. Which will need skipping if not available on the target. - regsub -- {RUNNABLE_PHOBOS_TEST.*$} $copy_line "" out_line + regsub -- {COMPILABLE_MATH_TEST.*$} $copy_line "" out_line set needs_phobos 1 } @@ -334,11 +373,17 @@ proc gdc-convert-test { base test } { puts $fdout "// { dg-additional-files \"$extra_files\" }" } + if { [llength $ddoc_options] > 0 } { + puts $fdout "// { dg-additional-options \"$ddoc_options\" }" + } + # Add specific options for test type # DMD's testsuite is extremely verbose, compiler messages from constructs # such as pragma(msg, ...) would otherwise cause tests to fail. - puts $fdout "// { dg-prune-output .* }" + if $saw_test_output { + puts $fdout "// { dg-prune-output .* }" + } # Compilable files are successful if an output is generated. # Fail compilable are successful if an output is not generated. @@ -357,6 +402,11 @@ proc gdc-convert-test { base test } { if !$saw_test_flags { set PERMUTE_ARGS $DEFAULT_DFLAGS } + # Tests mix C++ and D, but not all test options are common between + # the two front-end languages. + if { $type == "runnable_cxx" } { + puts $fdout "// { dg-prune-output \"'-frelease' is valid for D but not for C\" }" + } } compilable { @@ -384,6 +434,8 @@ proc gdc-convert-test { base test } { } } + gdc-extra-test-options $fdout $test + close $fdin close $fdout
Hi Iain, > Having a look, actually I can just use the presence of TEST_OUTPUT to be > a gate for whether to prune all output or not. that's quite nice indeed. > Maybe this can be improved later to extract the contents of TEST_OUTPUT, > but for now, it has caught a few hidden bugs in the tests that I have > handled in the patch below (no changelog entry yet). I've tried the patch myself last night on both i386-pc-solaris2.11 (both with as and gas) and sparc-sun-solaris2.11 (gas only until PR d/98584 is fixed) and found a few issues: > +proc gdc-extra-test-options { fdout test } { > + switch $test { [...] > + "runnable/test42.d" { > + # Tests that overflow line limits of older assemblers. > + puts $fdout "// { dg-xfail-if \"Lines exceed 10240 characters\" { *-*-solaris2.* && { ! gas } } }" This doesn't work: I get +XPASS: gdc.test/runnable/test42.d (test for excess errors) +UNRESOLVED: gdc.test/runnable/test42.d compilation failed to produce executable +XPASS: gdc.test/runnable/test42.d -shared-libphobos (test for excess errors) +UNRESOLVED: gdc.test/runnable/test42.d -shared-libphobos compilation failed to produce executable which is no wonder: due to the presence of TEST_OUTPUT in the test, all output gets pruned, including the assembler message. This leads to success myInt int myBool bool i s C6test42__T4T219TiZ1C C6test427test219FZ8__mixin11C Input string too long, limit 10240 compiler exited with status 1 XPASS: gdc.test/runnable/test42.d (test for excess errors) Unfortunately the exit status of compiler and assembler is ignored by the testsuite framework when deciding whether the compilation has succeeded. Besides, the dg-xfail-if above should be restricted to *86*-*-solaris2.* && ! gas (to allow for both i?86-*-solaris2.* and x86_64-*-solaris2.*): I've previously run a Solaris/SPARC build with as and libphobos enabled and only the 64-bit gdc.test/runnable/test42.d execution tests FAIL. While the Solaris sparc and x86 assemblers have a common origin, the code bases have diverged over time and the sparc assembler doesn't have this particular low limit. However, testing on Solaris/SPARC with gas and libphobos showed another issue: +FAIL: gdc.test/runnable/traits_getPointerBitmap.d (test for excess errors) UNRESOLVED: gdc.test/runnable/traits_getPointerBitmap.d compilation failed to produce executable +FAIL: gdc.test/runnable/traits_getPointerBitmap.d -shared-libphobos (test for excess errors) UNRESOLVED: gdc.test/runnable/traits_getPointerBitmap.d -shared-libphobos compilation failed to produce executable Excess errors: runnable/traits_getPointerBitmap.d:220:9: error: vector type __vector(float[4]) is not supported on this platform Rainer
diff --git a/libphobos/Makefile.in b/libphobos/Makefile.in index a1395929819..d42248405a2 100644 --- a/libphobos/Makefile.in +++ b/libphobos/Makefile.in @@ -15,7 +15,7 @@ @SET_MAKE@ # Makefile for the toplevel directory of the D Standard library. -# Copyright (C) 2006-2020 Free Software Foundation, Inc. +# Copyright (C) 2006-2021 Free Software Foundation, Inc. # # GCC is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/libphobos/configure b/libphobos/configure index a7fb5edb90f..d6e1d7463bb 100755 --- a/libphobos/configure +++ b/libphobos/configure @@ -15422,18 +15422,6 @@ $as_echo_n "checking for host support for libphobos... " >&6; } . ${srcdir}/configure.tgt case ${host} in x86_64-*-solaris2.* | i?86-*-solaris2.*) - # libphobos doesn't compile with the Solaris/x86 assembler due to a - # relatively low linelength limit. - as_prog=`$CC -print-prog-name=as` - if test -n "$as_prog" && $as_prog -v /dev/null 2>&1 | grep GNU > /dev/null 2>&1; then - druntime_cv_use_gas=yes; - else - druntime_cv_use_gas=no; - fi - rm -f a.out - if test x$druntime_cv_use_gas = xno; then - LIBPHOBOS_SUPPORTED=no - fi # 64-bit D execution fails with Solaris ld without -z relax=transtls support. if test "$druntime_ld_gld" = "no" && test "$druntime_ld_relax_transtls" = "no"; then LIBPHOBOS_SUPPORTED=no diff --git a/libphobos/configure.ac b/libphobos/configure.ac index cc9af29754f..254871f0a6c 100644 --- a/libphobos/configure.ac +++ b/libphobos/configure.ac @@ -185,18 +185,6 @@ AC_MSG_CHECKING([for host support for libphobos]) . ${srcdir}/configure.tgt case ${host} in x86_64-*-solaris2.* | i?86-*-solaris2.*) - # libphobos doesn't compile with the Solaris/x86 assembler due to a - # relatively low linelength limit. - as_prog=`$CC -print-prog-name=as` - if test -n "$as_prog" && $as_prog -v /dev/null 2>&1 | grep GNU > /dev/null 2>&1; then - druntime_cv_use_gas=yes; - else - druntime_cv_use_gas=no; - fi - rm -f a.out - if test x$druntime_cv_use_gas = xno; then - LIBPHOBOS_SUPPORTED=no - fi # 64-bit D execution fails with Solaris ld without -z relax=transtls support. if test "$druntime_ld_gld" = "no" && test "$druntime_ld_relax_transtls" = "no"; then LIBPHOBOS_SUPPORTED=no diff --git a/libphobos/libdruntime/Makefile.in b/libphobos/libdruntime/Makefile.in index 99ee8b92afa..1163207a138 100644 --- a/libphobos/libdruntime/Makefile.in +++ b/libphobos/libdruntime/Makefile.in @@ -15,7 +15,7 @@ @SET_MAKE@ # Makefile for the D runtime library. -# Copyright (C) 2012-2020 Free Software Foundation, Inc. +# Copyright (C) 2012-2021 Free Software Foundation, Inc. # # GCC is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by