[build,driver] RFC: Support compressed debug sections

"Joseph S. Myers" <joseph@codesourcery.com> writes:

> On Tue, 30 Apr 2013, Rainer Orth wrote:
>
>> 	* gcc.c (LINK_COMPRESS_DEBUG_SPEC, ASM_COMPRESS_DEBUG_SPEC):
>> 	Define.
>> 	(LINK_COMMAND_SPEC): Invoke LINK_COMPRESS_DEBUG_SPEC.
>> 	(asm_options): Invoke ASM_COMPRESS_DEBUG_SPEC.
>
> Note that there are separate copies of LINK_COMMAND_SPEC in darwin.h and 
> i386/djgpp.h, which maybe should include the new spec.

I hadn't thought about that.  Here's what I found:

* Darwin is Mach-O, but current gas supports DWARF-in-Mach-O.
  Unfortunately, current gcc mainline only supports Apple as:
  i386/darwin.h hardcodes as-only options in ASM_SPEC, which causes gas
  to barf.  If you manually invoke gas on gcc -gdwarf output, you get
  compressed debug sections, though objdump cannot handle them.

* DJGPP is COFF, but again gas seems to support DWARF.  Unfortunately,
  current gcc mainline doesn't even build:

gcc/config/i386/i386.c: In function 'void ix86_code_end()':
gcc/config/i386/i386.c:8667: error: 'ASM_DECLARE_FUNCTION_NAME' was not declared in this scope

  I didn't look further from here.

Anyway, I tried a i386-pc-solaris2.11 x i686-pc-darwin build with the
following patch to check it doesn't break the build:

	* config/darwin.h (LINK_COMMAND_SPEC_A): Invoke
	LINK_COMPRESS_DEBUG_SPEC.
	* config/i386/djgpp.h (LINK_COMMAND_SPEC): Likewise.
Especially darwin.h LINK_COMMAND_SPEC_A is a total mess: it's a copy of
gcc.c LINK_COMMAND_SPEC with various changes not carried forward and
other parts done in a slightly different syntax for no apparent reason.
I'm pretty sure the two could be merged to avoid problems from
duplication in the future, but I'll leave that to the Darwin
maintainers.

I can include the snippet above for symmetry, if only to reject -gz as
on ELF targets without the toolchain support, if you prefer.

> It's not clear to me from the documentation added by this patch whether 
> users are meant to specify their -gz options when compiling, when linking, 
> or both - and whether there might be cases when such an option is accepted 
> for one of compiling and linking but not the other (which would be 
> especially confusing).

The goal is to have -gz work for both compilation and linking, but
unfortunately, the current situation very much depends on both the
toolchain used and your intentions.  The following matrix shows read and
write support in the GNU and Solaris toolchains.  For the latter, full
support for zlib and zlib-gnu formats is being worked on, with ld
already there.

		GNU			Solaris
		gas  gld  gold	gdb	as   ld   dbx
		w    r w  r w	r	w    r w  r

      none	x    x x  x x	x	x    x x  x
      zlib-gnu	x    x 	  x x	x	     x x
      zlib	     	    		     x x

I plan to add zlib support first to gas and gdb, eventually to gld
(which also means adding support for compression on output, though I had
a very hard time finding my way through bfd and gld in the past), but
will rely on Cary Coutant and/or Ian, who were involved in the zlib/ELF
gABI format design, to handle gold.

Right now, the gas/gold combination supports -gz just fine (producing
the zlib-gnu format), while gas/gld only support -gz in the assemble
step.  I know this is highly undesirable, but whether or not it's
acceptable depends on your goals:

* If you mean to trade compilation/assembly speed for disk usage, -gz
  for compilation only is ok.

* If your primary goal is space savings for executables and shared
  objects while still allowing debugability, you need to use gold for
  -gz at link time to work.

	Rainer

I still have no idea from your answer how a user is meant to know whether 
to use the option when compiling, linking or both, which is what needs to 
be clear from invoke.texi.

What does it mean for the option to be supported for compiling but not 
linking?  What in that case will the linker do with compressed debug 
sections on input?  Combine them in some way, good or bad?  Uncompress 
them?

Likewise, for it to be supported for linking but not compiling?  Will the 
linker then compress the uncompressed sections it receives on input?

I think it would be better if the option semantics are more like "if you 
pass the same option when both compiling and linking, the linked output 
will have the sections appropriately compressed as specified by the 
option, whether or not the individual .o files do" - and if this can't be 
supported with the tools being used, don't allow the option.

Hi Joseph,

> I still have no idea from your answer how a user is meant to know whether 
> to use the option when compiling, linking or both, which is what needs to 
> be clear from invoke.texi.
>
> What does it mean for the option to be supported for compiling but not 
> linking?  What in that case will the linker do with compressed debug 
> sections on input?  Combine them in some way, good or bad?  Uncompress 
> them?

a linker that doesn't understand compressed debug sections will probably
combine them in a useless way.  If, like gld, it can read but not write
them, they will be written in uncompressed form.

> Likewise, for it to be supported for linking but not compiling?  Will the 
> linker then compress the uncompressed sections it receives on input?

Yes, that will work in all cases.

> I think it would be better if the option semantics are more like "if you 
> pass the same option when both compiling and linking, the linked output 
> will have the sections appropriately compressed as specified by the 
> option, whether or not the individual .o files do" - and if this can't be 
> supported with the tools being used, don't allow the option.

That's certainly an option, as in the following table (taking the
zlib-gnu format as an example, but omitting hypothetical linkers that
can write but not read compressed debug sections):

      assembler		linker		action			example
      w			r	w

      -			-	-	reject			as/ld
      -			x	-	reject, useless		as/gld
      -			x	x	ok, warn about no-op as -gz?	as/gold
      x			-	-	reject, harmful	   	gas/ld
      x			x	-	reject, useless		gas/gld
      x			x	x	ok			gas/gold

The only problem I see is that for an assembler not supporting
compression -gz would be a silent no-op.  I'd rather warn about this
instead, but still accept it so you can both compile and link with -gz.

	Rainer