Patchwork gengtype improvements for plugins, completed! patch 7/N [doc]

login
register
mail settings
Submitter Basile Starynkevitch
Date Sept. 11, 2010, 7:59 p.m.
Message ID <20100911195928.GA31368@hector.lesours>
Download mbox | patch
Permalink /patch/64529/
State New
Headers show

Comments

Basile Starynkevitch - Sept. 11, 2010, 7:59 p.m.
Hello All,

[join work by Basile Starynkevitch & Jeremie Salvucci]

References: 
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02060.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00616.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00663.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02063.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02065.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02068.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00983.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02069.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01029.html

The 7th and last part [doc] of our patch series "completed!" is a
documentation patch. I added a few missing sentences about some GTY
tags, and I documented imperfectly the new gengtype prorgram
interface, in particular its gengtype state facility.

Since I did not sent any patch on documentation, the difference is
only on gty.texi and has been obtained with

svn diff -x -p gcc/doc/gty.texi > ~/tmp/gengtype_patch_7_of_N__doc.diff

You can apply it to the trunk 163987 or to the previous patch 6/N [wstate].

As usual, I am attaching for convenience the cumulated patches against
trunk 163987 as a gzip file
gengtype_patch_7_of_N__doc-cumulatedto07-trunrev-163987.diff.gz

Jeremie did not write any documentation, so please blame (gently!) me
for the english mistakes etc, and suggest better phrasing.  

#################### gcc/ChangeLog entry for documentation 
2010-09-11  Basile Starynkevitch  <basile@starynkevitch.net>

	* gcc/doc/gty.texi:
	(Generating GGC code with gengtype): New node.
	(GTY Options): Documented that tag should be unique.
	Added ptr_alias documentation.
	(How to run the gengtype generator): New section.
################################################################

Comments are welcome.

Cheers.
Basile Starynkevitch - Sept. 11, 2010, 8:41 p.m.
Hello All,

[join work by Basile Starynkevitch & Jeremie Salvucci]

References: 
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02060.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00616.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00663.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02063.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02065.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02068.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00983.html
  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02069.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01029.html
  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01032.html


I did send a complete serie of patches for gengtype improvements for
plugins. I believe the code is working correctly for the required
functionality (ability to persist the entire state of gengtype by
serializing it into a textual file, and ability to reload, for
convenience of plugin developers.

Here are some questions related to my patch.

First, I do know that the first pieces of this patch series had
indentation issues. I will try to correct that.


Second, to all the kind people who did comment about our gengtype
work, could they check that I somehow adressed their comment (except
identation issues). I probably forgot some suggestions, but only by
mistake.  If you feel I did not address (either by explaining in an
email, or inside my patch proposal) some particular issue you did tell
me, please remind it kindly to me.


I do admit that I did not work with the right tools (eg quilt).

Laurynas suggested me to avoid backing up generated gt*.[ch] files
when they changed. I was thinking of adding a new program option to
gengtype, eg --backup or -B, so that
  gengtype -B .old .... [other gengtype options omitted]
would back-up every changed  generated gtFOO.h file into gtFOO.h.old
Is that acceptable?

A more practical issue, is where should the gtype.state file be
installed, under what file name, and how to do that? I have no good
idea and the installation part of gcc/Makefile.in makes me cry, I
don't understand most of it.

Should gengtype have a default state file to read in plugin mode? How
to wire-in such a default file?

How to have gengtype program installed? Under what name? gcc-gengtype?
Does the --program-suffix passed at ./configure time should matter (so
should it be gcc-gengtype-4.6 if ./configure-d with
--program-suffix=-4.6 or what)?  How to do that?  There are subtle
cross compilation issues I don't understand!!

I do hope some constructive comments on the patch series I just have
sent, in particular by reviewers able to later Ok (an improved version
of) it.

I will perhaps try to send a third round of this serie, very probably
in a single big email with 7 relative patches and ChangeLog entries.
Do you have practical advice on how to do that quickly?  So far, I
manually compared my version and hacked manually on a pristine copy of
the trunk, but that is very time-consuming (and my time budget on
gengtype is already negative!) I was thinking of starting with a more
recent trunk, applying each patch that I did send, resolving the
indentation issues...

I tried very hard to divide the work in several patches (and that
costed me a lot of work -including useless temporary workarounds-, it
would have been so much simpler to be able to send a single megapatch
with a very long message explaining all of it).  The constraint that
gengtype should work correctly & identically after every relative
patch is very harsh (and at the beginning of this patch serie
submission I was not even certain is was feasible).

Alternatively, if some kind person would be brave enough to start from
the patches we [= Jeremie & Basile] did send and improve them and
re-submit them we would be very happy; if you want to do that, please
tell us.

Again, if you did comment in the past on this gengtype work and you
feel I did not address or mention all of your comments, please
tell. Even better, a small summary of all your comments about this
gengtype work in a single email will be great.  I tried hard to follow
all the the suggestions (or at least to discuss them), but I am
unfortunately certain to have forgotten some (in particular because I
was painfully debugging gengtype-state.c while reading your comments).


What does the dreaded end of stage 1 means w.r.t. this gengtype
improvement?  Notice that by construction this improvement does not
change observably the behavior of gcc [or cc1 etc...], because
gengtype generates the same code in it!  So with or without this
gengtype improvement, the executables of GCC [I mean xgcc, cc1,
cc1plus, lto1, ...] do not change!

Regards.
Laurynas Biveinis - Sept. 13, 2010, 7:52 a.m.
+file. The @var{gtype.state} is usually in the plugin directory.

Double space before a new sentence, also might change when it is
finally decided where does that file actually go.

+The @code{gengtype} utility (which may be installed as
+@code{gcc-gengtype} or some other name on your system) is a C code

I guess this will be finalized when the name is settled but "or some
other name on your system)" is redundant

+generator program.  It generates suitable marking and type-walking C
+routines for GGC.  Generated GC marking routines ensure that every

"for GGC and PCH."

+live data is ultimately marked, so that GGC can delete, at the end of
+its @code{ggc_collect} function, every non-marked data (which is
+unreachable and dead).  Generated PCH type-walking routines are used
+for ``compilation'' of header files @var{*.h} into a @var{*.gch}
+file.

Looks redundant to me.


+The @code{gengtype} utility is run once specially when compiling GCC

I'd remove "specially"

+itself.  It parses many GCC source files (mostly internal GCC headers
+declaring @code{GTY}-ed types and internal GCC files declaring
+@code{GTY}-ed variables). It saves its state in a persistent textual

The comment in parentheses looks redundant to me.

+file @file{gtype.state}.  This file has a format tied to a particular
+GCC release and configuration.  This @file{gtype.state} file should be
+parsed only by @code{gengtype} (of the same version which has
+generated it).  Following GNU usage, @code{gengtype} accepts the usual

Do you enforce this in the code?


2010/9/11 Basile Starynkevitch <basile@starynkevitch.net>:
>
> Hello All,
>
> [join work by Basile Starynkevitch & Jeremie Salvucci]
>
> References:
>  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02060.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00616.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00663.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02063.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02065.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02068.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg00983.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-08/msg02069.html
>  http://gcc.gnu.org/ml/gcc-patches/2010-09/msg01029.html
>
> The 7th and last part [doc] of our patch series "completed!" is a
> documentation patch. I added a few missing sentences about some GTY
> tags, and I documented imperfectly the new gengtype prorgram
> interface, in particular its gengtype state facility.
>
> Since I did not sent any patch on documentation, the difference is
> only on gty.texi and has been obtained with
>
> svn diff -x -p gcc/doc/gty.texi > ~/tmp/gengtype_patch_7_of_N__doc.diff
>
> You can apply it to the trunk 163987 or to the previous patch 6/N [wstate].
>
> As usual, I am attaching for convenience the cumulated patches against
> trunk 163987 as a gzip file
> gengtype_patch_7_of_N__doc-cumulatedto07-trunrev-163987.diff.gz
>
> Jeremie did not write any documentation, so please blame (gently!) me
> for the english mistakes etc, and suggest better phrasing.
>
> #################### gcc/ChangeLog entry for documentation
> 2010-09-11  Basile Starynkevitch  <basile@starynkevitch.net>
>
>        * gcc/doc/gty.texi:
>        (Generating GGC code with gengtype): New node.
>        (GTY Options): Documented that tag should be unique.
>        Added ptr_alias documentation.
>        (How to run the gengtype generator): New section.
> ################################################################
>
> Comments are welcome.
>
> Cheers.
> --
> Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
> email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
> 8, rue de la Faiencerie, 92340 Bourg La Reine, France
> *** opinions {are only mines, sont seulement les miennes} ***
>
Laurynas Biveinis - Sept. 13, 2010, 7:58 a.m.
2010/9/11 Basile Starynkevitch <basile@starynkevitch.net>:
> Second, to all the kind people who did comment about our gengtype
> work, could they check that I somehow adressed their comment (except
> identation issues). I probably forgot some suggestions, but only by
> mistake.  If you feel I did not address (either by explaining in an
> email, or inside my patch proposal) some particular issue you did tell
> me, please remind it kindly to me.

I cross-checked against my previous comments, but I do think that the
burden is on you on this one. You can check the mailing list archives.

> Laurynas suggested me to avoid backing up generated gt*.[ch] files
> when they changed. I was thinking of adding a new program option to
> gengtype, eg --backup or -B, so that
>  gengtype -B .old .... [other gengtype options omitted]
> would back-up every changed  generated gtFOO.h file into gtFOO.h.old
> Is that acceptable?

I believe that since this is easily achievable by other means, such
option is not necessary.

> Should gengtype have a default state file to read in plugin mode? How
> to wire-in such a default file?

Why would you want to do that?

> How to have gengtype program installed? Under what name? gcc-gengtype?
> Does the --program-suffix passed at ./configure time should matter (so
> should it be gcc-gengtype-4.6 if ./configure-d with
> --program-suffix=-4.6 or what)?  How to do that?  There are subtle
> cross compilation issues I don't understand!!

I think the cross-compilation issues need to be worked out first (see
my other mail), and then answers to these questions will come more
naturally.

> I will perhaps try to send a third round of this serie, very probably
> in a single big email with 7 relative patches and ChangeLog entries.

I think it is better to have separate emails (if different folks will
want to comment on different patches, single e-mail thread will grow
unmanageable)

Patch

Index: incr-gengtype-gcc/gcc/doc/gty.texi
===================================================================
--- incr-gengtype-gcc/gcc/doc/gty.texi	(revision 163987)
+++ incr-gengtype-gcc/gcc/doc/gty.texi	(working copy)
@@ -69,6 +69,7 @@  These don't need to be marked.
 * GTY Options::         What goes inside a @code{GTY(())}.
 * GGC Roots::           Making global variables GGC roots.
 * Files::               How the generated files work.
+* Generating GGC code with gengtype::               How to run the gengtype generator.
 * Invoking the garbage collector::   How to invoke the garbage collector.
 @end menu
 
@@ -183,6 +184,7 @@  The value of the expression given by @code{desc} i
 each @code{tag} value, each of which should be different.  If no
 @code{tag} is matched, the field marked with @code{default} is used if
 there is one, otherwise no field in the union will be marked.
+A union field can have at most one @code{tag}.
 
 In the @code{desc} option, the ``current structure'' is the union that
 it discriminates.  Use @code{%1} to mean the structure containing it.
@@ -312,6 +314,12 @@  and the @var{from expression} returns the original
 the pointer.  The pointer will be available using the @code{%h}
 escape.
 
+@findex ptr_alias
+@item ptr_alias (@var{type})
+
+When applied to a type, @code{ptr_alias} indicates that the GTY-ed
+type is an alias or synonym of the given @var{type}.
+
 @findex chain_next
 @findex chain_prev
 @findex chain_circular
@@ -472,12 +480,55 @@  somewhere.  It will be called @file{gtype-@var{lan
 @var{lang} is the name of the subdirectory the language is contained in.
 
 Plugins can add additional root tables.  Run the @code{gengtype}
-utility in plugin mode as @code{gengtype -P pluginout.h @var{source-dir}
-@var{file-list} @var{plugin*.c}} with your plugin files
-@var{plugin*.c} using @code{GTY} to generate the @var{pluginout.h} file.
-The GCC build tree is needed to be present in that mode.
+utility in plugin mode as @code{gengtype -P @var{gt-pluginout.h} -r
+@var{gtype.state} @var{plugin*.c}} with your plugin files
+@var{plugin*.c} using @code{GTY} to generate the @var{gt-pluginout.h}
+file. The @var{gtype.state} is usually in the plugin directory.
 
 
+@node Generating GGC code with gengtype
+@section How to run the gengtype generator.
+@cindex garbage collector, generation, gengtype
+@findex gengtype
+
+The @code{gengtype} utility (which may be installed as
+@code{gcc-gengtype} or some other name on your system) is a C code
+generator program.  It generates suitable marking and type-walking C
+routines for GGC.  Generated GC marking routines ensure that every
+live data is ultimately marked, so that GGC can delete, at the end of
+its @code{ggc_collect} function, every non-marked data (which is
+unreachable and dead).  Generated PCH type-walking routines are used
+for ``compilation'' of header files @var{*.h} into a @var{*.gch}
+file.  The @code{gengtype} utility is used when building GCC itself
+from its source tree, and also for plugins dealing with their own
+GTY-ed data.
+
+The @code{gengtype} utility is run once specially when compiling GCC
+itself.  It parses many GCC source files (mostly internal GCC headers
+declaring @code{GTY}-ed types and internal GCC files declaring
+@code{GTY}-ed variables). It saves its state in a persistent textual
+file @file{gtype.state}.  This file has a format tied to a particular
+GCC release and configuration.  This @file{gtype.state} file should be
+parsed only by @code{gengtype} (of the same version which has
+generated it).  Following GNU usage, @code{gengtype} accepts the usual
+@code{--version} or @code{-V} and @code{--help} arguments.  It also
+accepts @code{--verbose} or @code{-v} to show what it is doing, and
+@code{--debug} or @code{-D} for debugging purposes. It may dump its
+state in human readable form with @code{--dump} or @code{-d}.
+
+During GCC build process, a file @file{gtyp-input.list} is built and
+contains a long list of files to be parsed.  Then, @code{gengtype} is
+run once to generate its state with @code{gengtype -S
+@file{source-directory} -I @file{gtyp-input.list} -w
+@file{gtype.state}}. Immediately after, it is re-run to read its just
+generated state and generate @file{gt*.[ch]} files with @code{gengtype
+-r @file{gtype.state}}. 
+
+Plugin developers using @code{GTY} in their plugin source code should
+run @code{gengtype -P @var{gt-pluginout.h} -r @file{gtype.state}
+@var{plugin*.c}}
+
+
 @node Invoking the garbage collector
 @section How to invoke the garbage collector
 @cindex garbage collector, invocation