[Darwin] proposal for a 4.6 work-around to dsymutil calling issues.

Hi,

PR43751 and PR46749 show that we have a problem with the existing way  
in which dsymutil is called by trying to recognize that the c/l  
contains a source file.

Not only do we not cover all source types, but the whole thing breaks  
down with relatively benign changes in the c/l.

AFAICT, we don't really have a suitable facility to:
'determine that a source is present for the set of supported languages  
and insert a flag'

On the basis that such a wide-ranging addition might not be  
appropriate for stage-3
- - I'm going to propose a work-around and suggest an enhancement bug  
for 4.7?

Of course, if someone knows a better solution .. great!

The change is this:

When the user wants a dSYM (s)he will now need to append "-dsym" to  
the c/l - instead of gcc figuring it out automatically.
If there are no existing "-g*" on the c/l "-g" will be appended.

I have not yet written doc for this - pending comments on whether it's  
considered a reasonable work-around for 4.6.

thoughts?
Iain


  #define DARWIN_LIBSYSTEM_HAS_UNWIND

On Dec 2, 2010, at 3:45 AM, IainS wrote:
> PR43751 and PR46749 show that we have a problem with the existing way in which dsymutil is called by trying to recognize that the c/l contains a source file.

So, you propose to just remove the feature and break one of the fundamental usage models of compilation on unix environments, because....

Help me flesh out the remainder of that sentence.

> Not only do we not cover all source types,

We could add support for more source types, if we cared.  Initially Apple only did the C family of languages because that was product and dsymutil wasn't tested with other languages (and might not work).

> but the whole thing breaks down with relatively benign changes in the c/l.

And those bugs I'd posit, can be fixed.

> AFAICT, we don't really have a suitable facility to:
> 'determine that a source is present for the set of supported languages and insert a flag'

When the compilation backend (cc1, cc1plus) of a language is called, trivially, a source is present.  gcc.c knows when a source is present, because it calls the compiler.  Further, it know when it is creating an intermediate .o file that will then be removed during compilation.  Is there any information it doesn't have?

> I have not yet written doc for this - pending comments on whether it's considered a reasonable work-around for 4.6.

Nope...  I prefer the status quo over completely breaking the unix compilation model.

Hi Mike, All...

On 2 Dec 2010, at 19:59, Mike Stump wrote:

> On Dec 2, 2010, at 3:45 AM, IainS wrote:
>> PR43751 and PR46749 show that we have a problem with the existing  
>> way in which dsymutil is called by trying to recognize that the c/l  
>> contains a source file.
>
> So, you propose to just remove the feature and break one of the  
> fundamental usage models of compilation on unix environments,  
> because....

... I proposed to provide an alternative .. but .. I will finish the  
sentence anyway ;-) .....

> Help me flesh out the remainder of that sentence.

... it doesn't work reliably or consistently (with the current compiler)

gcc test.c -o hc -g3 ==>  produces a dSYM
gcc test.c -o hc -g3 -lmylib ==> does not.

(I don't think that is obvious)

with 'c' one can fix it ...
gcc -o hc -g3 -lmylib test.c

(again... not terribly obvious - and that solution doesn't work for  
Fortran)

A whole raft of innocuous (and programatically necessary) c/l changes  
seem to precipitate this.
There isn't any particular evidence that there is a fault in the  
suffix parsing - it lies elsewhere
(and there appears to be no guarantee that drivers will not alter it
   - e.g. the recent change to fortran to append the quadmath spec has  
stopped it working with any c/l ordering I've tried
    [with the Fortran file suffixes added to the list, of course]).

... it's not that the non-working command lines are wrong or buggy  
either.

>> Not only do we not cover all source types,
>
> We could add support for more source types, if we cared.  Initially  
> Apple only did the C family of languages because that was product  
> and dsymutil wasn't tested with other languages (and might not work).

well, I've experimented with that:
- I suspect that we have no (more) problems adding (e.g.) Fortran than  
the ones we already know about with dsymutil and current c/debug.

- you know from previous attempts that enabling dsymutil for the  
Fortran suite will result in a whole load of spurious fallout (hence  
the hack wrapper from x months back).    However, the errors appear to  
be just that - spurious - and the dSYM seems reasonably OK.

- for Java I've only succeeded in debugging with "-save-temps" ... the  
dSYM seg-faults the installed gdb on Darwin9 & 10.

>> but the whole thing breaks down with relatively benign changes in  
>> the c/l.
>
> And those bugs I'd posit, can be fixed.

Indeed - I don't doubt it  - my conjecture was that to do so might  
well be too invasive for stage 3 (however, that is conjecture).

>> AFAICT, we don't really have a suitable facility to:
>> 'determine that a source is present for the set of supported  
>> languages and insert a flag'
>
> When the compilation backend (cc1, cc1plus) of a language is called,  
> trivially, a source is present.  gcc.c knows when a source is  
> present, because it calls the compiler.  Further, it know when it is  
> creating an intermediate .o file that will then be removed during  
> compilation.  Is there any information it doesn't have?

-- I've tried a whole bunch of non-invasive, darwin-local approaches  
with specs [most recently with driver specs] without success.
  (unless those have been fixed within the last few days) ***

>> I have not yet written doc for this - pending comments on whether  
>> it's considered a reasonable work-around for 4.6.
> Nope...  I prefer the status quo over completely breaking the unix  
> compilation model.


well, my view is that it's already at least partially broken, and/or  
inconsistent for the current suite of gcc languages.

However, that statement doesn't represent disagreement...
... it's was, and remains, on the TODO [PR43751] (although someone  
else is welcome to try).

If we {cannot,do not have time to} fix this within 4.6 ...
... I would suggest that a documented, working, facility where the  
user puts "-dsym" on the c/l to get a dSYM is better than something  
which works part-time (and not at all for Fortran)  ---  after all,  
the user has to put -g* to get debug output anyway -- it's not like  
we're asking for something radically different.

cheers
Iain

*** this is actually possible  - Richi just fixed a related bug.

On Dec 2, 2010, at 1:12 PM, IainS wrote:
> On 2 Dec 2010, at 19:59, Mike Stump wrote:
> 
>> On Dec 2, 2010, at 3:45 AM, IainS wrote:
>>> PR43751 and PR46749 show that we have a problem with the existing way in which dsymutil is called by trying to recognize that the c/l contains a source file.
>> 
>> So, you propose to just remove the feature and break one of the fundamental usage models of compilation on unix environments, because....
> 
> ... I proposed to provide an alternative .. but .. I will finish the sentence anyway ;-) .....

> ... it doesn't work reliably or consistently (with the current compiler)

I disagree.  I think the proper direction would be to fix the bugs, so that it works 100% instead of 90%.

> gcc test.c -o hc -g3 ==>  produces a dSYM
> gcc test.c -o hc -g3 -lmylib ==> does not.
> 
> (I don't think that is obvious)

$ ls -lt DEL.dSYM
ls: DEL.dSYM: No such file or directory
$ ./xgcc -B./ t.c -o DEL -g3 -L. -lmylib
$ ls -lt DEL.dSYM
total 0
drwxr-xr-x  4 mrs  mrs  136 Dec  2 13:26 Contents

?  Now, I don't doubt there are bugs, but at least for me the above doesn't seem to be one of them.

> - I suspect that we have no (more) problems adding (e.g.) Fortran than the ones we already know about with dsymutil and current c/debug.

If you run the fortran test suite with the  %{.c|.cc|.C|.cpp|.cp|.c++|.cxx|.CPP|.m|.mm|.s: condition removed from darwin9.h, are there no regressions?  Does debugging appear to then work in fortran?  If so, let's add the fortran suffixes to the list.  We know it is necessary, though, maybe not sufficient.

I just tried it in my tree, it did exactly what I expected.  It ran dsymutil, then it unlinked the .o files.  I've added a comment in the PR for you to add a testcase, as trivially, I don't yet have it.  Maybe this issue if just lto related?  I'm in a non-lto tree, currently.

> - you know from previous attempts that enabling dsymutil for the Fortran suite will result in a whole load of spurious fallout (hence the hack wrapper from x months back).    However, the errors appear to be just that - spurious - and the dSYM seems reasonably OK.
> 
> - for Java I've only succeeded in debugging with "-save-temps" ... the dSYM seg-faults the installed gdb on Darwin9 & 10.

Well, lots of possibilities exist:

Then either, we should recommend FSF gdb (if it work) for java, and/or bug report it to Apple, or try llvm's debugger on the grounds that Apple is unlikely to continue forward with gdb and/or use stabs with java...  Lots of possibilities.

> well, my view is that it's already at least partially broken, and/or inconsistent for the current suite of gcc languages.

Yes, that may well be the case, but to break it further doesn't help the people that use it reliably today.

> If we {cannot,do not have time to} fix this within 4.6 ...
> ... I would suggest that a documented, working, facility where the user puts "-dsym" on the c/l to get a dSYM is better than something which works part-time (and not at all for Fortran)  ---  after all, the user has to put -g* to get debug output anyway -- it's not like we're asking for something radically different.

I don't favor adding flags and documentation for hack arounds.  I'd rather just fix the real problem.  I have yet to have someone even say what the real problem is.  I suspect lto is just a little bit wrong.

Hi Mike,

On 2 Dec 2010, at 22:36, Mike Stump wrote:

> ?  Now, I don't doubt there are bugs, but at least for me the above  
> doesn't seem to be one of them.

actually, as of current trunk (likely 167292) the spec-processing for  
suffixes appears to be fixed (good timing eh?).

>> - I suspect that we have no (more) problems adding (e.g.) Fortran  
>> than the ones we already know about with dsymutil and current c/ 
>> debug.
>
> If you run the fortran test suite with the  %{.c|.cc|.C|.cpp|.cp|.c+ 
> +|.cxx|.CPP|.m|.mm|.s: condition removed from darwin9.h, are there  
> no regressions?  Does debugging appear to then work in fortran?  If  
> so, let's add the fortran suffixes to the list.  We know it is  
> necessary, though, maybe not sufficient.

we know that dsymutil produces a number of (stated by the dsymutil  
maintainers to be spurious & harmless) warnings.
These will cause Fortran test-suite regressions unless handled
    (I produced a wrapper script to filter them, if you recall -  
attached to comment #4 of PR43751)
    or we could put 'prune' statements in the relevant tests.

Although the script is a hack, it does have the advantage that it is  
one point of fixing rather than many - and that it can be easily  
removed if/when the dsymutil problems are resolved.

---

Anyway, that's for a different patch/thread  - this issue has 'gone  
away' ...

Iain

On Fri, Dec 03, 2010 at 09:20:44AM +0000, IainS wrote:
> Hi Mike,
>
> On 2 Dec 2010, at 22:36, Mike Stump wrote:
>
>> ?  Now, I don't doubt there are bugs, but at least for me the above  
>> doesn't seem to be one of them.
>
> actually, as of current trunk (likely 167292) the spec-processing for  
> suffixes appears to be fixed (good timing eh?).
>
>>> - I suspect that we have no (more) problems adding (e.g.) Fortran  
>>> than the ones we already know about with dsymutil and current c/ 
>>> debug.
>>
>> If you run the fortran test suite with the  %{.c|.cc|.C|.cpp|.cp|.c+ 
>> +|.cxx|.CPP|.m|.mm|.s: condition removed from darwin9.h, are there no 
>> regressions?  Does debugging appear to then work in fortran?  If so, 
>> let's add the fortran suffixes to the list.  We know it is necessary, 
>> though, maybe not sufficient.
>
> we know that dsymutil produces a number of (stated by the dsymutil  
> maintainers to be spurious & harmless) warnings.
> These will cause Fortran test-suite regressions unless handled
>    (I produced a wrapper script to filter them, if you recall - attached 
> to comment #4 of PR43751)
>    or we could put 'prune' statements in the relevant tests.

Exactly how many different dsymutil warnings do you run into with gfortran?
The one described in PR43254 is fixed in Xcode 3.2.3 so darwin10 and later
shouldn't show these problems now. 
            Jack

>
> Although the script is a hack, it does have the advantage that it is one 
> point of fixing rather than many - and that it can be easily removed 
> if/when the dsymutil problems are resolved.
>
> ---
>
> Anyway, that's for a different patch/thread  - this issue has 'gone  
> away' ...
>
> Iain
>
>