| Message ID | 4D9D56F4.3050203@gmail.com |
|---|---|
| State | New |
| Headers | show |
On Thu, Apr 7, 2011 at 8:17 AM, Laurynas Biveinis <laurynas.biveinis@gmail.com> wrote: > Fixes a bunch of C testsuite failures. Committed to gc-improv. > > 2011-04-07 Laurynas Biveinis <laurynas.biveinis@gmail.com> > > * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. That looks strange, labels should be function specific, except non-local labels. Maybe you can get away with allocating DECL_NONLOCAL labels on the permanent rtl obstack? Perhaps a third, per-translation-unit obstack is necessary? Ciao! Steven
2011/4/8 Steven Bosscher <stevenb.gcc@gmail.com>: >> * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. > > That looks strange, labels should be function specific, except > non-local labels. Maybe you can get away with allocating DECL_NONLOCAL > labels on the permanent rtl obstack? That's a good idea, I will try it once things are stable on the branch. > Perhaps a third, per-translation-unit obstack is necessary? Perhaps. After I finish with permanent rtl obstack, I will measure how large it gets and if it's worthwhile to split out per-translation-obstack out of it. Thanks,
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/08/11 07:21, Laurynas Biveinis wrote: > 2011/4/8 Steven Bosscher <stevenb.gcc@gmail.com>: >>> * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. >> >> That looks strange, labels should be function specific, except >> non-local labels. Maybe you can get away with allocating DECL_NONLOCAL >> labels on the permanent rtl obstack? > > That's a good idea, I will try it once things are stable on the branch. > >> Perhaps a third, per-translation-unit obstack is necessary? > > Perhaps. After I finish with permanent rtl obstack, I will measure how > large it gets and if it's worthwhile to split out > per-translation-obstack out of it. And then you'll want a per-statement obstack, then per-expression obstack, and before you know it, GCC looks much like it did 20 years ago. Sigh. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNnx1iAAoJEBRtltQi2kC7YJwIAI02Q9vDzyqmvcZGe25jb93A /esh3wWlDSV8TKKwBJkRt6D8PDlRqqaNa5owcEm0iKAYN3lcBo+Q0nYGqoYnN5b1 YVIiajfR809L2NeQOCjVwYpZotQa0MOGuWRrpIRYRCMEOFgljGzwa1utj5qOUXEr s27d0vNnE5ShuJAOt+uJlDU9xgaNTHrCcJBwacKGBBxKRba8nV2wQ9uJPjypoAeX Fza9uJQRqe9rmOheUZsPIthNeZbUUCb0jVzGbivUzpa6gyLQgLry66IvnuAg7BPP Xo2lQh3CDZY3QJnktuiDA201RvPRlxhoA+8jTfMMPi+/NexS3+GrLA9fTC+QkE0= =hQs1 -----END PGP SIGNATURE-----
On Fri, Apr 8, 2011 at 4:36 PM, Jeff Law <law@redhat.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 04/08/11 07:21, Laurynas Biveinis wrote: >> 2011/4/8 Steven Bosscher <stevenb.gcc@gmail.com>: >>>> * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. >>> >>> That looks strange, labels should be function specific, except >>> non-local labels. Maybe you can get away with allocating DECL_NONLOCAL >>> labels on the permanent rtl obstack? >> >> That's a good idea, I will try it once things are stable on the branch. >> >>> Perhaps a third, per-translation-unit obstack is necessary? >> >> Perhaps. After I finish with permanent rtl obstack, I will measure how >> large it gets and if it's worthwhile to split out >> per-translation-obstack out of it. > And then you'll want a per-statement obstack, then per-expression > obstack, and before you know it, GCC looks much like it did 20 years ago. > > Sigh. I remember it was fast at that time though ;) Richard.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/08/11 08:39, Richard Guenther wrote: > On Fri, Apr 8, 2011 at 4:36 PM, Jeff Law <law@redhat.com> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 04/08/11 07:21, Laurynas Biveinis wrote: >>> 2011/4/8 Steven Bosscher <stevenb.gcc@gmail.com>: >>>>> * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. >>>> >>>> That looks strange, labels should be function specific, except >>>> non-local labels. Maybe you can get away with allocating DECL_NONLOCAL >>>> labels on the permanent rtl obstack? >>> >>> That's a good idea, I will try it once things are stable on the branch. >>> >>>> Perhaps a third, per-translation-unit obstack is necessary? >>> >>> Perhaps. After I finish with permanent rtl obstack, I will measure how >>> large it gets and if it's worthwhile to split out >>> per-translation-obstack out of it. >> And then you'll want a per-statement obstack, then per-expression >> obstack, and before you know it, GCC looks much like it did 20 years ago. >> >> Sigh. > > I remember it was fast at that time though ;) Yea, fast to segfault dereferencing a dangling pointer due to placing an object on the wrong obstack. And fast because we didn't have a good way to keep most expressions beyond the lifetime of a single statement thus preventing any kind of higher level analysis and optimization. Jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNnx7OAAoJEBRtltQi2kC7cLQH/RDU86grC6BTnVc9iG5KE4zu B+YP0+42dJfrinjL0cqkPeEprTr5j144XJYujsxT49f7G+qu1qX+gzyfowgTdOwb dIfhnWXoC9d09E0nXD+mLVwHwNwsrkv1loN501wryPksSOsgjxjhwc7TCW8WnW93 2sjrpKLg0qvy+/8KH0RTA80q9xehGkfqux3y4ZbNLsPBreYp+jhTzsVqjxk+T0F7 Ges5PFLjXwQdbx+iFBStQhlZHf8A/3TRXnrfllPRRl37bzR56Jnza2ag8EkVHwXi 5bNQ+YWqJXHiKG2nInARTNGnJXQU8XMGYYaZTx1Myl0c8osAlfu0QwLHrJn+Few= =4X+7 -----END PGP SIGNATURE-----
On Fri, Apr 8, 2011 at 4:36 PM, Jeff Law <law@redhat.com> wrote: >>> Perhaps a third, per-translation-unit obstack is necessary? >> >> Perhaps. After I finish with permanent rtl obstack, I will measure how >> large it gets and if it's worthwhile to split out >> per-translation-obstack out of it. > And then you'll want a per-statement obstack, then per-expression > obstack, and before you know it, GCC looks much like it did 20 years ago. I don't think that's a fair comment, or a real concern. Unlike 20 years ago, GCC now has a reasonable idea of how long RTX objects live. I can think of only four states for RTX in memory: 1. RTL that lives throughout the compilation, things like shared constants. RTX objects in this group are only in GC memory now because one RTX object on GC-memory cannot point to a non-GC RTX object (the mark-and-sweep process expects all objects to be in GC memory). 2. RTL from back-end (re-)initialization. Stuff in this group should go on a permanent obstack that is only removed if the back end is re-initialized (e.g. for the MIPS back end, or for future multi-target support). 3. RTL per translation unit. I am not sure what falls in this class. Things like RTL for global variables maybe? RTL that may escape from functions? Apparently the latter happens with labels, I hope it only happens with nested functions, really. Got some insights to share here? ;-) 4. RTL per function. GCC expands one GIMPLE function at a time, and the idea is to initialize the RTL obstack once when expanding starts, let it grow until final, and blow it away after final. Unlike 20 years ago, this obstack is never rolled back during RTL passes. This relies on generating not too much garbage, but memory for per-function RTL should be dwarfed by per-translation unit GIMPLE anyway. Jeff, few people from 20 years ago are still around, so perhaps you can help a bit to help make things are going in the right direction. ;-) Ciao! Steven
2011/4/8 Jeff Law <law@redhat.com>: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 04/08/11 07:21, Laurynas Biveinis wrote: >> 2011/4/8 Steven Bosscher <stevenb.gcc@gmail.com>: >>>> * stmt.c (label_rtx): Allocate RTX in permanent RTL memory. >>> >>> That looks strange, labels should be function specific, except >>> non-local labels. Maybe you can get away with allocating DECL_NONLOCAL >>> labels on the permanent rtl obstack? >> >> That's a good idea, I will try it once things are stable on the branch. >> >>> Perhaps a third, per-translation-unit obstack is necessary? >> >> Perhaps. After I finish with permanent rtl obstack, I will measure how >> large it gets and if it's worthwhile to split out >> per-translation-obstack out of it. > And then you'll want a per-statement obstack, then per-expression > obstack, and before you know it, GCC looks much like it did 20 years ago. It is certainly true that moving away from GC will make some kinds of bugs possible again, but I hope that not enough to be an unmanageable concern. The RTL object lifetimes seem to be clear in most of the instances and so far I am going with only two of them: permanent and function. After the initial conversion is done, I don't expect much trouble for any new RTL allocations introduced to be decided which memory area they belong to. Adding a third area, e.g. per-TU, of course would complicate this, but I still believe this is manageable. -- Laurynas
2011/4/9 Steven Bosscher <stevenb.gcc@gmail.com>: > 4. RTL per function. GCC expands one GIMPLE function at a time, and > the idea is to initialize the RTL obstack once when expanding starts, > let it grow until final, and blow it away after final. Unlike 20 years > ago, this obstack is never rolled back during RTL passes. This relies > on generating not too much garbage, but memory for per-function RTL > should be dwarfed by per-translation unit GIMPLE anyway. Well, I have plans to see if it is worthwhile for pass like combine to rollback the function obstack to do away with scratch RTL. Of course this depends, on how much memory can be saved by doing this - in comparison to current GC.
On Sun, 10 Apr 2011 21:27:10 +0300 Laurynas Biveinis <laurynas.biveinis@gmail.com> wrote: > 2011/4/9 Steven Bosscher <stevenb.gcc@gmail.com>: > > 4. RTL per function. GCC expands one GIMPLE function at a time, and > > the idea is to initialize the RTL obstack once when expanding starts, > > let it grow until final, and blow it away after final. Unlike 20 years > > ago, this obstack is never rolled back during RTL passes. This relies > > on generating not too much garbage, but memory for per-function RTL > > should be dwarfed by per-translation unit GIMPLE anyway. > > Well, I have plans to see if it is worthwhile for pass like combine to > rollback the function obstack to do away with scratch RTL. Of course > this depends, on how much memory can be saved by doing this - in > comparison to current GC. I respect a lot Laurynas' work, but my general personal feeling & wish is on the contrary that in the long term, more GCC data should be garbage collected, and that GCC's garbage collector should be better. However, I tend to believe that Laurynas cleanup on RTL could be helpful. And in the long run, I would imagine that making RTL data garbage collectable again would just be a matter of adding GTY annotations somewhere. Regards.
On Sun, Apr 10, 2011 at 8:22 PM, Laurynas Biveinis <laurynas.biveinis@gmail.com> wrote: > It is certainly true that moving away from GC will make some kinds of > bugs possible again, but I hope that not enough to be an unmanageable > concern. One thing that may help, is to poison parts of released obstacks, instead of actually releasing the memory. A bit like the poison mechanism of GGC. An obstack memory checking mechanism similar to gcac would also be nice. Something that walks all RTL and verifies that RTX objects are allocated on the obstacks where you'd expect them to be (via _obstack_allocated_p). I don't think GCC had any checking like that 20 years ago... Ciao! Steven
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/10/11 12:27, Laurynas Biveinis wrote: > 2011/4/9 Steven Bosscher <stevenb.gcc@gmail.com>: >> 4. RTL per function. GCC expands one GIMPLE function at a time, and >> the idea is to initialize the RTL obstack once when expanding starts, >> let it grow until final, and blow it away after final. Unlike 20 years >> ago, this obstack is never rolled back during RTL passes. This relies >> on generating not too much garbage, but memory for per-function RTL >> should be dwarfed by per-translation unit GIMPLE anyway. > > Well, I have plans to see if it is worthwhile for pass like combine to > rollback the function obstack to do away with scratch RTL. Of course > this depends, on how much memory can be saved by doing this - in > comparison to current GC. One of the fundamental problems you have to watch out for when dealing with scratch objects is how to handle the case when you belatedly realize you want the object to have a longer lifetime. The obvious solution is you copy the object, but then you have to be able to distinguish within the object, what fields point to other temporary objects vs permanent objects so that you can copy the referenced temporary objects, but not the permanent objects (other parts of the compiler may expect those permanent objects to be unique/shared). Not fun, not at all fun. Been there, done that. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNo150AAoJEBRtltQi2kC7Yb0H+gPB2ub86sbkGx0ee5ry1YYc ww222sMb+YP6wQ/fIxi/tYXdfxcouJ4/SuiC03tYNAwvfONZuQNrZKyEwu5cPXIh OMQYhV5pxDLfaRmpklBZWfYndStlWUYrmZAHPLI0zO5BCxgQaiZx/A6zjg6lPNY/ VnMKpdF1Tp0M03tJ1JNqMlTKrP5mkV/gAsjQVyjAM1DJntLYIqxqmm4tinaAJXUf pYDHecQgV/ZvngzCI8XydNZXEk/GnrcVCnyByO1BBLOzCom63+WXzXAGE9HPTOvj fNncNhOFz5rAzowiddngkoxlnPNGJKbuypprt/U5u17j91MlGADRDTkcapvI21A= =PwyX -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/10/11 12:22, Laurynas Biveinis wrote: > > It is certainly true that moving away from GC will make some kinds of > bugs possible again, but I hope that not enough to be an unmanageable > concern. It's definitely a huge concern. More for tree objects than RTL objects though. The RTL object lifetimes seem to be clear in most of the > instances and so far I am going with only two of them: permanent and > function. After the initial conversion is done, I don't expect much > trouble for any new RTL allocations introduced to be decided which > memory area they belong to. > > Adding a third area, e.g. per-TU, of course would complicate this, but > I still believe this is manageable. So what's the plan for the case where you need to change the lifetime of an object? What's the plan for building some kind of consistency checking to ensure that we aren't referencing dangling pointers. I ask these questions because they were a serious source of problems in the past and any significant revamping of memory management needs to have a reasonable story for how to deal with them, else we're taking a rather significant step backwards. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNo1+YAAoJEBRtltQi2kC7LeMH/2pxPMnlJAsjiwHApURV/sxX 5XyGBvawPs1W6zobBRUeOhIrfn8hSm6P2QywZvj7EpfTbiD/aKdVXz8zBOC2J1IG 69TDQwXY4YhqEW5WxTsBK6YAoFTALebZe6dbLRuN5795X+d5rSZmlyiX/GgICB7M 2iMqqkH6kv9wO2k6pfeN6k+hIHZmpVHRg3KeADTWvO5+3FKkVWFizA3LHhPf/pDM 7sG5o6CB8AI7PBNgh6A7xNs045NexIhEdkQ/R7jQNpySk3XHpfOPhjKh135hWwnw 6UoR8xqned5nr1sj6n07i+hSvYDLT6Izm68ZnFe5E09lPemVe4rZnO6Sb/+fhOA= =gD9u -----END PGP SIGNATURE-----
On Apr 11, 2011, at 1:03 PM, Jeff Law wrote: > The obvious solution is you copy the object, but then you have to be > able to distinguish within the object, what fields point to other > temporary objects vs permanent objects so that you can copy the > referenced temporary objects, but not the permanent objects (other parts > of the compiler may expect those permanent objects to be unique/shared). > Not fun, not at all fun. Been there, done that. And then someone tells you that you can't copy... Usually a nasty bug report much later. So the choice is then, not solve a problem, or make everything permanent or add GC (back). :-) Been there, done that too.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/11/11 18:21, Mike Stump wrote: > On Apr 11, 2011, at 1:03 PM, Jeff Law wrote: >> The obvious solution is you copy the object, but then you have to be >> able to distinguish within the object, what fields point to other >> temporary objects vs permanent objects so that you can copy the >> referenced temporary objects, but not the permanent objects (other parts >> of the compiler may expect those permanent objects to be unique/shared). >> Not fun, not at all fun. Been there, done that. > > And then someone tells you that you can't copy... Usually a nasty bug report much later. So the choice is then, not solve a problem, or make everything permanent or add GC (back). :-) Been there, done that too. Right. Hence the old hack to mark the temporary obstacks and make them permanent (wasting an enormous amount of space in the process). I don't recall the name of that function, but it certainly made me barf. And yes, I remember all too well the problems with deep copying having written one of the earliest tree deep copy routines to support - -fcombine-statics . It was the source of more problems than I could ever count -- all related to objects allocated to different obstacks hanging off a single tree node or objects of two different types being stored into the same field within tree nodes. That was circa 1992/1993. These were the kind of problems that ultimately led to the GC system we have today. There are clearly things that can be better handled with different allocation models and I'll support moving to better models where it makes sense. Where it doesn't, obviously I won't support it. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNo77qAAoJEBRtltQi2kC7X0UH/0n1ydyhcvfjjoqhQCIwjf9J jp3MOel6I1RTjAMm69N0ZqCP5t8AuHIIJdlrSQ1Aworx0gNl4+VjbEXJEM6dcQB4 enJ5mYmPXQ3EMDEi6C/uPIGwPcPsqO9sFTe91plQIsI7m6OTRjM4I/aS9XmRQ/uQ 8qBhDTSY3sTn4rOTUvqKfvdStP9B/Sf2tYpYbHgm/tfLRQ5UFHWpZFckcyntHFs8 rTxcA3xKlVNSg2D9+3CfM8KVWqMVXeHgB+tlB2Q6L7/TAeCqGHABE5MxeB1+HDJN QoB0HCMCt7t6sqGYk6S5xayyfJUrWvt5XBiVa8oXrP9sVh1iNM8mY33zzwPE2M4= =9inU -----END PGP SIGNATURE-----
On Tue, Apr 12, 2011 at 4:54 AM, Jeff Law <law@redhat.com> wrote: > On 04/11/11 18:21, Mike Stump wrote: >> On Apr 11, 2011, at 1:03 PM, Jeff Law wrote: >>> The obvious solution is you copy the object, but then you have to be >>> able to distinguish within the object, what fields point to other >>> temporary objects vs permanent objects so that you can copy the >>> referenced temporary objects, but not the permanent objects (other parts >>> of the compiler may expect those permanent objects to be unique/shared). >>> Not fun, not at all fun. Been there, done that. >> >> And then someone tells you that you can't copy... Usually a nasty bug report much later. So the choice is then, not solve a problem, or make everything permanent or add GC (back). :-) Been there, done that too. > Right. Hence the old hack to mark the temporary obstacks and make them > permanent (wasting an enormous amount of space in the process). I don't > recall the name of that function, but it certainly made me barf. > > And yes, I remember all too well the problems with deep copying having > written one of the earliest tree deep copy routines to support > - -fcombine-statics . It was the source of more problems than I could > ever count -- all related to objects allocated to different obstacks > hanging off a single tree node or objects of two different types being > stored into the same field within tree nodes. That was circa 1992/1993. > > > These were the kind of problems that ultimately led to the GC system we > have today. There are clearly things that can be better handled with > different allocation models and I'll support moving to better models > where it makes sense. Where it doesn't, obviously I won't support it. I am unsure how to interpret that last sentence. Do you currently believe that the approach of 2 or 3 obstacks for RTL is a better model that makes sense, or not? I think all these comments from you "old guys" ;-) are more discouraging than fair. What Laurynas and Bernd have done, is nothing like the old obstacks of GCC2. I am also a bit surprised that all these fears only come up now that someone actually tries to do what Zack already suggested in 2003, around the time that unit-at-a-time compilation became the default mode. Anyway, please do give this idea a chance at least. There is a big difference between the pre-GC obstacks and what Laurynas is now trying (not just in terms of memory management, but in the entire compilation process!) and apart from the name "obstacks" this idea and the pre-GC memory management in GCC have nothing in common. Ciao! Steven
On Tue, Apr 12, 2011 at 08:33:56AM +0200, Steven Bosscher wrote: > I think all these comments from you "old guys" ;-) are more > discouraging than fair. What Laurynas and Bernd have done, is nothing It is IMHO completely fair to point that the risks this brings in a huge maintainance nightmare are very high. Jakub
On Tue, Apr 12, 2011 at 9:01 AM, Jakub Jelinek <jakub@redhat.com> wrote: > On Tue, Apr 12, 2011 at 08:33:56AM +0200, Steven Bosscher wrote: >> I think all these comments from you "old guys" ;-) are more >> discouraging than fair. What Laurynas and Bernd have done, is nothing > > It is IMHO completely fair to point that the risks this brings in > a huge maintainance nightmare are very high. And IM-equally-HO it is completely unfair to talk about risks in any situation where there is nothing yet to talk about! Give it a chance and wait for something that's more than just an idea, and then assess the risks based on an implementation. Or just say "this won't fly" now so that people who would like to work on this can turn their attention to something else. Also fine. Really. Ciao! Steven
On Apr 12, 2011, at 1:45 AM, Steven Bosscher wrote: > On Tue, Apr 12, 2011 at 9:01 AM, Jakub Jelinek <jakub@redhat.com> wrote: >> On Tue, Apr 12, 2011 at 08:33:56AM +0200, Steven Bosscher wrote: >>> I think all these comments from you "old guys" ;-) are more >>> discouraging than fair. What Laurynas and Bernd have done, is nothing >> >> It is IMHO completely fair to point that the risks this brings in >> a huge maintainance nightmare are very high. > > And IM-equally-HO it is completely unfair to talk about risks in any > situation where there is nothing yet to talk about! Give it a chance > and wait for something that's more than just an idea, and then assess > the risks based on an implementation. The problem is that in the olden days, people thought it would work, then, they ran into, oh, but the lifetime is wrong. Oh, but I don't know what the lifetime will be before I start allocating. Oh, what do you mean I can't copy it... All these things happened after real bug reports and great difficulty in tracking down real problems. A review might not be able to spot the things that won't work. We don't want to discourage, only to let everyone know what our experience is with these sorts of problems. So, for example, if one is predicating copying objects to solve lifetime problems, we can expound what some of the problems were in the past. They could be relevant, they might not be not relevant; also, they just might cause people to explore an area that might be a problem, find and address it sooner. Also, actual patches have been posted since Jan 27th... I don't see that all the problems are cleverly avoided. So, let's take one: > - If objects stored in PCH have pointers pointing outside of PCH-able/GC-managed memory, these become wild pointers on PCH read even with GTY((skip)) applied properly. However, not all GTY((skip)) pointers point outside of PCH-able memory, so I have overloaded GTY((deletable)) option to reset such fields to NULL on PCH write. This is only a part of the fix as logic needs to be reviewed what to do with these NULLs after the PCH read. In case of DECL_RTL, the NULL in the field causes re-creation of rtx anew, which I think is the right thing. Not an example of a warm fuzzy. When you combine this with the entire idea of PCH, and that is to slosh to disk and back, the entire state of the compiler, essentially, unmodified, we run into a conceptual gap. Now, why did PCH have that design, so that we could limit the number of pch bugs and come up with a robust design that just works. This is a useful property. Now, one can audit DECL_RTL, to see if it matters, but there are roughly 344 places to check. I glanced at a few, to see if I can spot problems. I find it hard to know if any particular one will be a problem. At least some of them can't be found by testing. Presently we do what I'd call smoke testing for pch. Deeper issues aren't covered. So, who then will spot the issues?
On Tue, Apr 12, 2011 at 12:44 PM, Mike Stump <mikestump@comcast.net> wrote: >> - If objects stored in PCH have pointers pointing outside of PCH-able/GC-managed memory, these become wild pointers on PCH read even with GTY((skip)) applied properly. However, not all GTY((skip)) pointers point outside of PCH-able memory, so I have overloaded GTY((deletable)) option to reset such fields to NULL on PCH write. This is only a part of the fix as logic needs to be reviewed what to do with these NULLs after the PCH read. In case of DECL_RTL, the NULL in the field causes re-creation of rtx anew, which I think is the right thing. > > Not an example of a warm fuzzy. When you combine this with the entire idea of PCH, and that is to slosh to disk and back, the entire state of the compiler, essentially, unmodified, we run into a conceptual gap. Now, why did PCH have that design, so that we could limit the number of pch bugs and come up with a robust design that just works. This is a useful property. I've debuged PCH bugs and they were usually a _result_ of the poor design. Missing GTY on non-pointer objects for example. There is nothing robust about the design of PCH and it certainly does not "just work". But, two things: 1. RTL and PCH should be completely separate. No DECL_RTL should ever end up in a PCH because DECL_RTL is not created in the front end (or should not be, it'd be a bug) 2. PCH as we know it will probably be gone soon (Google's pph stuff) Ciao! Steven
> So what's the plan for the case where you need to change the lifetime of > an object? Copying it. Frankly at the moment I don't how much trouble does deep copying from scratch to function entails, as mentioned in your other e-mail. ATM I am working at separating permanent from function. If it turns out to be unsurmountable, then, well, lesson learned. > What's the plan for building some kind of consistency > checking to ensure that we aren't referencing dangling pointers. I haven't thought them out in detail at this moment, but I had some ideas, very close to what Steven suggested: poisoning RTL obstacks instead of freeing/reusing. Walking existing GC memory at RTL function memory freeing time to see that no pointers point to it. The latter is virtually free even. > I ask these questions because they were a serious source of problems in > the past and any significant revamping of memory management needs to > have a reasonable story for how to deal with them, else we're taking a > rather significant step backwards. I plan to propose merging the branch once such concerns will be adequately addressed. Personally I believe that it is possible to address them. In the case it is impossible to address them, I will propose merging very lite version of the branch that changes RTL allocation interface, keeps GC behind it, and leaves the rest for presumably brighter future. Hopefully this will not be the case. Regarding PCH: so far whenever I touch it, frankly I hate it. In the current context: the RTL output there was early-generated RTL which IMHO itself is a sign of a poor IL separation in GCC. So in that respect I believe my work is a step in a right direction, even if painful. Thanks for your feedback,
[Resending with the correct Mike's address, sorry for the spam] > So what's the plan for the case where you need to change the lifetime of > an object? Copying it. Frankly at the moment I don't how much trouble does deep copying from scratch to function entails, as mentioned in your other e-mail. ATM I am working at separating permanent from function. If it turns out to be unsurmountable, then, well, lesson learned. > What's the plan for building some kind of consistency > checking to ensure that we aren't referencing dangling pointers. I haven't thought them out in detail at this moment, but I had some ideas, very close to what Steven suggested: poisoning RTL obstacks instead of freeing/reusing. Walking existing GC memory at RTL function memory freeing time to see that no pointers point to it. The latter is virtually free even. > I ask these questions because they were a serious source of problems in > the past and any significant revamping of memory management needs to > have a reasonable story for how to deal with them, else we're taking a > rather significant step backwards. I plan to propose merging the branch once such concerns will be adequately addressed. Personally I believe that it is possible to address them. In the case it is impossible to address them, I will propose merging very lite version of the branch that changes RTL allocation interface, keeps GC behind it, and leaves the rest for presumably brighter future. Hopefully this will not be the case. Regarding PCH: so far whenever I touch it, frankly I hate it. In the current context: the RTL output there was early-generated RTL which IMHO itself is a sign of a poor IL separation in GCC. So in that respect I believe my work is a step in a right direction, even if painful. Thanks for your feedback, -- Laurynas
On 04/11/2011 10:03 PM, Jeff Law wrote: > One of the fundamental problems you have to watch out for when dealing > with scratch objects is how to handle the case when you belatedly > realize you want the object to have a longer lifetime. Historically, our problems with obstacks were a consequence of line-by-line processing of the input file and never being quite certain where a given object would have to end up. Almost all of that is simply no longer an issue, which means RTL memory management is conceptually a lot simpler than it used to be. A few things (constants, mostly) are shared and must be permanent, the rest can go a single per-function obstack which is freed after every function. I don't think we should (or need to) try to have more obstacks for RTL. I also don't see why a deep copy to the permanent obstack would be a source of problems these days. More common than wanting to do that however is the opposite case where you can speculatively generate RTL on the function obstack, see if it's valid, and if not reset the obstack to the earlier point to reclaim the memory. I know that reload and combine could be made more efficient this way, and I'd guess the same is true for fwprop. I'd also like to point out that I share Steven's experience that the GC stuff doesn't "just work"; I've certainly had objects collected from under me and needed to puzzle out where to sprinkle the necessary GTY markers. Bernd
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/12/11 02:45, Steven Bosscher wrote: > On Tue, Apr 12, 2011 at 9:01 AM, Jakub Jelinek <jakub@redhat.com> wrote: >> On Tue, Apr 12, 2011 at 08:33:56AM +0200, Steven Bosscher wrote: >>> I think all these comments from you "old guys" ;-) are more >>> discouraging than fair. What Laurynas and Bernd have done, is nothing >> >> It is IMHO completely fair to point that the risks this brings in >> a huge maintainance nightmare are very high. > > And IM-equally-HO it is completely unfair to talk about risks in any > situation where there is nothing yet to talk about! Give it a chance > and wait for something that's more than just an idea, and then assess > the risks based on an implementation. Given that we've already got a goodly amount of experience with obstack and GC based mechanisms for allocation, I think it is completely fair and wise to discuss the known risk/reward for both. > > Or just say "this won't fly" now so that people who would like to work > on this can turn their attention to something else. Also fine. Really. I have serious concerns about reverting to obstacks as our main memory allocation approach for tree & rtl data. However, I also realize that there are objects where obstacks make sense and I realize some of those objects may currently be hanging off tree or RTL structures. With that in mind, I'm all for a critical examination of our data structures, their lifetimes and what allocation model works best. From that I would expect that we'll find some cases where an obstack model works better and the structures will move to that model. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNpGmEAAoJEBRtltQi2kC7cjUIALtwCRbcsxABIx6xtWlZIHRy vPfB8pc6u+IhIrbu/T2qZjUoP6bq5UQIgCPIy3o7qSgJ5Qd8NvYJQ5nMHMyPZvX2 MDzyTJcMB1OUsoZhgVwTdZoL1aSp3ARopruDM2DNc9zo8DXP5YFcn2w6bXaASjr5 jENRoiqTe6hLgJXQZT7QQObusR6gM4Off78Hs/vGlaOmeXMSONfMTxms1ya0ROQe h+YyXpQuLsUDdIO9wSHfeD0/73er8fLYqlcJ77GPUDK907oVtr4bKUWOirwX9QL2 vRwMur93cVjPvHuNUPZdNxsrpozJH+G/iAIPJVa3K+AYXBvvzWtSpDHb4ttvy48= =pN2J -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/12/11 05:54, Bernd Schmidt wrote: > On 04/11/2011 10:03 PM, Jeff Law wrote: >> One of the fundamental problems you have to watch out for when dealing >> with scratch objects is how to handle the case when you belatedly >> realize you want the object to have a longer lifetime. > > Historically, our problems with obstacks were a consequence of > line-by-line processing of the input file and never being quite certain > where a given object would have to end up. That was certainly the cause of the bulk of the problems. And to be honest, the multi-obstack design made sense when it was written, assuming one could keep track of which obstack was the right one to use. As we wanted to solve new problems, the obstack approach really fell apart. Also note that trees were a *much* bigger problem than RTL WRT memory management. That's not to say there weren't any RTL problems, but that we ran into far more problems with tree allocations. Almost all of that is simply > no longer an issue, which means RTL memory management is conceptually a > lot simpler than it used to be. In some respects, yes, memory management in general is conceptually a hell of a lot simpler. I'll note that some of the simplifications came as a side effect of moving to GC allocation, but aren't inherent to using GC allocation. For example, removal of storing RTL into tree structures without a suitable union :-) In other respects I think memory management has become more difficult, at least to change because usage of GC has allowed us to allocate and mostly forget. One can certainly argue that model has problems, but I would claim the problems with allocate and forget were smaller than the problems we had with explicit management via obstacks. A few things (constants, mostly) are > shared and must be permanent, the rest can go a single per-function > obstack which is freed after every function. I don't think we should (or > need to) try to have more obstacks for RTL. To have two obstacks for RTL, you really have to have a bright line that everyone knows and understands and that you can codify as well so that things like deep copying can DTRT. If you can't codify it for a deep copy, then, IMHO, it's a non-starter. Also note that having verification capabilities helps considerably. You also need rules for if/when/how a single RTL object might validly reference RTL objects with differing lifetimes. > > I also don't see why a deep copy to the permanent obstack would be a > source of problems these days. For RTL, the biggest problem is making sure you do the right thing with RTL that is expected to be unique & shared (trees are much more difficult), which as you note is primarily going to be constants and the like. More common than wanting to do that > however is the opposite case where you can speculatively generate RTL on > the function obstack, see if it's valid, and if not reset the obstack to > the earlier point to reclaim the memory. I know that reload and combine > could be made more efficient this way, and I'd guess the same is true > for fwprop. There are definitely things that obstacks handle well, I won't dispute that. > > I'd also like to point out that I share Steven's experience that the GC > stuff doesn't "just work"; I've certainly had objects collected from > under me and needed to puzzle out where to sprinkle the necessary GTY > markers. In my experience, I've seen far fewer allocation mistakes with the GC system than with the multi-obstack system we had prior to the introduction of GC. Neither is going to be 100% perfect as programmers can certainly goof things up. The question is which is easier to deal with in the general case and I'd say GC wins by a wide margin. Of course, that comes with a penalty (performance), thus I'm open to exploring what objects really need to be part of the GC system and which can be handled by other allocation schemes such as obstacks. jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNpHAeAAoJEBRtltQi2kC7HpQIAIchzQ/uBei7FaIJIbOK7dFn kOey7eqhySdb1WTY4mMKfPPRrqD9rCfR1PPftH/x4250lRMLTKgN3WvCtUmSef8o tAM4hwViadQUvOlJPjIve3VqNg2fV6CmG/mtnChuwAknsUe7z/xm+C+J6lPd0Mts LWOqya+JGYrGVuSP9+BdANEVUM4mkoWaZ2pXOzml7pkvW2JQBloaJLlyN7lobShQ Kw2wTnpzVlnyeCT8Ow8AVHk1Yw4osZkKmN5VrjRmGaKaOtl2CeQk+LODFmHp+YrE Jkc/YTNtYev6Pt30QfE26qylcjWqJte3me0Ux5J8KR5fxNc2NFz+Vs20m6zy+M4= =q93W -----END PGP SIGNATURE-----
On Apr 12, 2011, at 4:46 AM, Laurynas Biveinis wrote: >> So what's the plan for the case where you need to change the lifetime of >> an object? > > Copying it. Frankly at the moment I don't how much trouble does deep > copying from scratch to function entails, The code to copy isn't too hard and if fairly simple to understand, test and do. Roughly, you just need to understand how deep you have to copy, part of this is ensuring you iterator over all the pointers in the objects that might point to the shorter lifetime. The problems of the past would be, all the other folks that have live pointers into anything which is copied. You have the first pointer and that is trivial to change, but all the other pointers could be problematic. Some of the problems were when some of the pointers pointed to the new and some of the pointers pointed to the old, then, for all the data hanging off those data structures that were supposed to be single pieces of data, there were now two copies of that data. Imagine if you had two TREE_ASM_WRITTENs for a `single' decl. RTL is vastly simpler than a _DECL node. For just generated RTL (combine temp rtl building), I would not expect any problems. Likewise scratch to function, those should be easy enough. Harder are function to anything longer, say, permanent.
Index: stmt.c =================================================================== --- stmt.c (revision 170593) +++ stmt.c (working copy) @@ -139,7 +139,10 @@ if (!DECL_RTL_SET_P (label)) { - rtx r = gen_label_rtx (); + rtx r; + use_rtl_permanent_mem (); + r = gen_label_rtx (); + use_rtl_function_mem (); SET_DECL_RTL (label, r); if (FORCED_LABEL (label) || DECL_NONLOCAL (label)) LABEL_PRESERVE_P (r) = 1;