More ObjC++ work on @encode in C++ templates

This patch completes the work to get @encode working in C++ templates.
I had to finish it myself as there was nothing more to merge from the
apple/trunk branch on the matter. :-)

Ok to apply ?

Thanks

On Oct 7, 2010, at 11:30 AM, Nicola Pero wrote:
> This patch completes the work to get @encode working in C++ templates.
> I had to finish it myself as there was nothing more to merge from the
> apple/trunk branch on the matter. :-)
> 
> Ok to apply ?

Ok.

In general, please try and avoid collisions on the testcase names with those names from llvm-gcc-4.2...  If they have an encode-10.mm testcase, but not an encode-15, go for encode-15.  The reason is that once we are done with the basics, one can then see which testcases are in that tree to see what features and functionalities we are still missing.  We may well have to reimplement the feature, but at least we can have a quick pointer to what we are missing.

> In general, please try and avoid collisions on the testcase names with those names 
> from llvm-gcc-4.2...  If they have an encode-10.mm testcase, but not an encode-15, 
> go for encode-15.  The reason is that once we are done with the basics, one can 
> then see which testcases are in that tree to see what features and functionalities 
> we are still missing.  We may well have to reimplement the feature, but at least 
> we can have a quick pointer to what we are missing.

Sure.

Alternatively, maybe we should add new testcases (that we're not backporting from
the apple/trunk branch) to a new directory ?  Then we would never have an issue with 
conflicts. :-)

I think IainS was reorganizing the ObjC/ObjC++ testcases, maybe he has a recommendation
of where to add new testcases ?

Thanks

On 7 Oct 2010, at 23:33, Nicola Pero wrote:

>
>> In general, please try and avoid collisions on the testcase names  
>> with those names
>> from llvm-gcc-4.2...  If they have an encode-10.mm testcase, but  
>> not an encode-15,
>> go for encode-15.  The reason is that once we are done with the  
>> basics, one can
>> then see which testcases are in that tree to see what features and  
>> functionalities
>> we are still missing.  We may well have to reimplement the feature,  
>> but at least
>> we can have a quick pointer to what we are missing.
>
> Sure.
>
> Alternatively, maybe we should add new testcases (that we're not  
> backporting from
> the apple/trunk branch) to a new directory ?  Then we would never  
> have an issue with
> conflicts. :-)
>
> I think IainS was reorganizing the ObjC/ObjC++ testcases, maybe he  
> has a recommendation
> of where to add new testcases ?

I can see the merit in making it easy to see what we've still got left  
to do ...
... and it makes good sense to use distinct names for new tests [maybe  
even put fsf at the front?]

... but it would be really really nice to avoid one huge dir that does  
all tests (where the differentiation in topic is solely by filename)...

Recently, I have been trying to add new stuff (where there are more  
than a few tests - or expected to be more than a few tests) to a new  
topic subdir from obj*.dg/
[e.g tls/, attributes/ .. and soon property/ ...  (I would prob. have  
added foreach/ ;-)) ]
Similarly, we can do that for torture tests under obj*.dg/torture/

we are a small group - so it shouldn't be too hard to debate what  
merits a separate category.

one then needs to identify all the tests in an area from LLVM and  
compare with the same area in our suite
- that is a little work - but relatively small work c.f. doing the  
coding.

IMO the long-term benefit of organizing the tests in our suite  
outweighs that little work in identifying the set of tests for a given  
topic in the LLVM structure.
(some form of tracking of additions/changes would be beneficial too -  
but I don't know that we've got enough resources to implement that).

---

I'd also like:
a/ to move the old objc/{compile,execute} to dg ... it is more  
flexible and makes ObjC++ consistent w/ObjC
b/ to  figure out a method for doing ObjC* common tests as is done  
with c/c++  - there are a lot of duplicates that could be merged.
c/ make sure that we are torture-testing enough stuff (e.g. ObJC++ had  
no optimized tests until recently, which was hiding a global fail).
d/ make one sub-structure for each runtime [and make sure we've  
covered the code-gen] since we're heading for 4 runtimes to test ;-)

just my £0.02...
Iain


P.S. even if it were feasible license-wise (which currently looks  
unlikely) - one cannot simply symlink to the LLVM dir and run the tests
-- there is too much divergence in the testsuite infrastructure.

PPS I guess we can always use find to flatten our dir structure and  
LLVM's and compare file lists that way.

> ... but it would be really really nice to avoid one huge dir that does  
> all tests (where the differentiation in topic is solely by filename)...
> 
> Recently, I have been trying to add new stuff (where there are more  
> than a few tests - or expected to be more than a few tests) to a new  
> topic subdir from obj*.dg/
> [e.g tls/, attributes/ .. and soon property/ ...  (I would prob. have  
> added foreach/ ;-)) ]
> Similarly, we can do that for torture tests under obj*.dg/torture/

Sure, I love the idea of using subdirectories.  We could put all the "old"
tests into a "misc" or "old" subdirectory.  So, we reorganize tests as in:

 objc.dg/misc - all the "old" tests that are in common with llvm-gcc-4.2 so Mike
(and everyone else) can easily do the test comparison he wants to do.  Tests we
backport from apple/trunk branch would end in here.

 objc.dg/fast_enumeration - all the new fast enumeration tests

 objc.dg/encode - all the new encode tests

 etc. all new tests would go into an appropriate subdirectory

Over time we'd end up with something similar to testsuite/g++.dg/ where they 
have testcases neatly organized in subdirectories.



> I'd also like:
> a/ to move the old objc/{compile,execute} to dg ... it is more  
> flexible and makes ObjC++ consistent w/ObjC
> b/ to  figure out a method for doing ObjC* common tests as is done  
> with c/c++  - there are a lot of duplicates that could be merged.
> c/ make sure that we are torture-testing enough stuff (e.g. ObJC++ had  
> no optimized tests until recently, which was hiding a global fail).
> d/ make one sub-structure for each runtime [and make sure we've  
> covered the code-gen] since we're heading for 4 runtimes to test ;-)

Sounds good.


> P.S. even if it were feasible license-wise (which currently looks  
> unlikely) - one cannot simply symlink to the LLVM dir and run the tests
> -- there is too much divergence in the testsuite infrastructure.

I think Mike was referring to llvm-gcc-4.2, which looks like having the 
same testsuite as we do - just with more ObjC/ObjC++ tests.  And I'd love 
to take our ObjC/ObjC++ compiler and run it on the ObjC/ObjC++ testsuite 
of another compiler to look for bugs or features to implement. ;-)

On 8 Oct 2010, at 08:43, Nicola Pero wrote:

>
>> ... but it would be really really nice to avoid one huge dir that  
>> does
>> all tests (where the differentiation in topic is solely by  
>> filename)...
>>
>> Recently, I have been trying to add new stuff (where there are more
>> than a few tests - or expected to be more than a few tests) to a new
>> topic subdir from obj*.dg/
>> [e.g tls/, attributes/ .. and soon property/ ...  (I would prob. have
>> added foreach/ ;-)) ]
>> Similarly, we can do that for torture tests under obj*.dg/torture/
>
> Sure, I love the idea of using subdirectories.  We could put all the  
> "old"
> tests into a "misc" or "old" subdirectory.  So, we reorganize tests  
> as in

we would, in any case, end up needing a misc dir - since it is  
inefficient to have only one or two tests in a subdir (all the  
overhead of a new .exp).

> objc.dg/misc - all the "old" tests that are in common with llvm- 
> gcc-4.2 so Mike
> (and everyone else) can easily do the test comparison he wants to  
> do.  Tests we
> backport from apple/trunk branch would end in here.

I don't favor duplicating tests - and IMO, in the end, for both old  
and new developers it will be easier to look for all tests for  
'encode' in a sub-dir named 'encode' ... ;-)
[even if the dev. knew that the tests used to live in /objc.dg ]

> objc.dg/fast_enumeration - all the new fast enumeration tests
>
> objc.dg/encode - all the new encode tests

great :-)

> etc. all new tests would go into an appropriate subdirectory

hmmmm  ....
I'd favor keeping all 'topic-x' tests together --  that's what helps  
the dev. to find what's been tested and what hasn't ...

.... and to allow comparison with external sources - adopt a  
convention that differentiates new from old & imported.
(maybe agree to prefix new tests with "fsf-" ?   --   so that we don't  
need to worry about name clashes with future LLVM tests)

I doubt we can import LLVM tests, for the same reason we can't import  
the code -- so the only tests to be 'imported' are from the FSF apple/ 
trunk branch.

We can keep the same names for old, and apple-local imports -  
therefore a simple 'find' stripping the path prefixes should be able  
to flatten both our tree and LLVM making comparison easy.

>> P.S. even if it were feasible license-wise (which currently looks
>> unlikely) - one cannot simply symlink to the LLVM dir and run the  
>> tests
>> -- there is too much divergence in the testsuite infrastructure.
>
> I think Mike was referring to llvm-gcc-4.2, which looks like having  
> the
> same testsuite as we do - just with more ObjC/ObjC++ tests.  And I'd  
> love
> to take our ObjC/ObjC++ compiler and run it on the ObjC/ObjC++  
> testsuite
> of another compiler to look for bugs or features to implement. ;-)

it's not just the layout --
  -- the functionality of lib/*.exp has diverged and some of the  
mechanisms expected from 4.2 era tests don't work with 4.6 era  
realities.

(IIRC the Makefiles don't do quite the same things -- e.g. there is no  
ability to do check-gcc-objc from the top level).

cheers,
Iain

>> Sure, I love the idea of using subdirectories.  We could put all the  
>> "old"
>> tests into a "misc" or "old" subdirectory.  So, we reorganize tests  
>> as in
>
> we would, in any case, end up needing a misc dir - since it is  
> inefficient to have only one or two tests in a subdir (all the  
> overhead of a new .exp).

I briefly looked at g++.dg/ and they have a single dg.exp which runs
all the tests in all the subdirectories (with the exceptions of the directories
that run special tests).

So there should be no particular overhead to using subdirectories; even multiple
levels. :-)

So, maybe we could use

 objc.dg/encode/base/... <encode tests shared with the apple/trunk branch>
 objc.dg/encode/...      <new encode tests>

 objc.dg/fast-enumeration/base/... <fast enumeration tests shared with the apple/trunk branch>
 objc.dg/fast-enumeration/...      <new fast enumeration tests>

 objc.dg/{topic}/base/...          <test on {topic} shared with the apple/trunk branch>
 objc.dg/{topic}/...               <new tests on {topic} that we add>

That should take care of your concern that all tests for fast enumeration (for example)
should be in the same place.

It would still be easy to compare our list of tests with the one from llvm-gcc-4.2 (as Mike
wants to be able to), as you would simply do

ls objc.dg/*/base/

to get the list of our tests shared with the apple/trunk branch (you don't even need find), which
you can immediately compare with the list in llvm-gcc-4.2 (and, there won' be any confusing clashes
due to names).

Anyway, let's continue the discussion when you are back online in a few days. :-)

Thanks

On Oct 8, 2010, at 12:43 AM, Nicola Pero wrote:
> Sure, I love the idea of using subdirectories.  We could put all the "old"
> tests into a "misc" or "old" subdirectory.

old is never a good name.  Neither is new.

The usual convention is to leave the testsuite as is and not to `reorganize' it.  There is little benefit from the reorganization and the benefit of being able to compare and track failures and regressions with stable testcase names is at time useful.  Also, using the stable names to be able to compare with llvm-gcc-4.2 is useful.  I'd like to see that preserved.  Now, if we didn't have any functionality remaining to be merged, and there were 0 testcase failures on all platforms, the benefit of the stable names lessons, but, I don't think we are there yet.

> The usual convention is to leave the testsuite as is and not to `reorganize' it.

I agree that we have more pressing things to do with Objective-C than reorganizing
the testsuite :-)

I still think it would be worth doing it (there are some very practical benefits), 
but I understand you may feel it's a waste of time at this stage and, considering
there are only few weeks left before the end of stage 1, I agree and won't insist
or spend more time discussing it. ;-)

Let's ignore it and maybe pick it up at the next release.

Thanks