[2/4] docs: add build infrastructure for gtkdocs

By convention, documented headers now go in include/

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 Makefile      |    6 +++++-
 Makefile.docs |   15 +++++++++++++++
 QEMU-docs.xml |   31 +++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 1 deletions(-)
 create mode 100644 Makefile.docs
 create mode 100644 QEMU-docs.xml

On 12/14/2011 06:20 PM, Anthony Liguori wrote:
> By convention, documented headers now go in include/

Dislike.

> +include $(SRC_PATH)/Makefile.docs
> +
>  $(common-obj-y): $(GENERATED_HEADERS)
>  subdir-libcacard: $(oslib-obj-y) $(trace-obj-y) qemu-timer-common.o
>  
> @@ -113,6 +115,8 @@ QEMU_CFLAGS+=$(CURL_CFLAGS)
>  
>  QEMU_CFLAGS+=$(GLIB_CFLAGS)
>  
> +QEMU_CFLAGS+=$(SRC_PATH)/include
> +
>  ui/cocoa.o: ui/cocoa.m

Documentation should be built by default, so that errors in the format
are detected (and break the build).

>  
> +
> +gtkdoc: html/index.html
> +
> +html/index.html: $(DOC_SRC)
> +	gtkdoc-scan --module=QEMU --source-dir=$(SRC_PATH)/include && \
> +        cp $(SRC_PATH)/QEMU-docs.xml . && \
> +	gtkdoc-mkdb --module=QEMU --output-format=xml --source-dir=$(SRC_PATH)/include && \
> +	mkdir -p html && \
> +        (cd html && gtkdoc-mkhtml QEMU ../QEMU-docs.xml && cd ..) && \
> +	gtkdoc-fixxref --module=QEMU --module-dir=html
> +

Does this thing not support incremental builds?

> +++ b/QEMU-docs.xml
> @@ -0,0 +1,31 @@
> +<?xml version="1.0"?>
> +<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
> +               "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"
> +[
> +  <!ENTITY % local.common.attrib "xmlns:xi  CDATA  #FIXED 'http://www.w3.org/2003/XInclude'">
> +]>
> +<book id="index">
> +  <bookinfo>
> +    <title>QEMU Reference Manual</title>
> +    <releaseinfo>
> +      for QEMU 1.0.

1.1.

Am 15.12.2011 10:37, schrieb Avi Kivity:
> On 12/14/2011 06:20 PM, Anthony Liguori wrote:
>> By convention, documented headers now go in include/
>
> Dislike.
>> +include $(SRC_PATH)/Makefile.docs
>> +
>> $(common-obj-y): $(GENERATED_HEADERS)
>> subdir-libcacard: $(oslib-obj-y) $(trace-obj-y) qemu-timer-common.o
>>
>> @@ -113,6 +115,8 @@ QEMU_CFLAGS+=$(CURL_CFLAGS)
>>
>> QEMU_CFLAGS+=$(GLIB_CFLAGS)
>>
>> +QEMU_CFLAGS+=$(SRC_PATH)/include
>> +
>> ui/cocoa.o: ui/cocoa.m
>
> Documentation should be built by default, so that errors in the format
> are detected (and break the build).

I agree. It is built by default today, and there is a configure option
to disable this default behavior.

>>
>> +
>> +gtkdoc: html/index.html
>> +
>> +html/index.html: $(DOC_SRC)
>> +	gtkdoc-scan --module=QEMU --source-dir=$(SRC_PATH)/include&&  \
>> +        cp $(SRC_PATH)/QEMU-docs.xml .&&  \
>> +	gtkdoc-mkdb --module=QEMU --output-format=xml --source-dir=$(SRC_PATH)/include&&  \
>> +	mkdir -p html&&  \
>> +        (cd html&&  gtkdoc-mkhtml QEMU ../QEMU-docs.xml&&  cd ..)&&  \
>> +	gtkdoc-fixxref --module=QEMU --module-dir=html
>> +
>
> Does this thing not support incremental builds?

I don't know the answer for gtkdoc, but at least for doxygen incremental 
builds
are not useful if you want to use advanced features like cross reference 
listings:
bidirectional references must parse all files (or would need rather 
complicated
dependency lists).

Parsing of the QEMU code and generation of advanced html output (which
includes style sheets and graphics) with doxygen takes about 4 min on my 
server
and creates 70 MB of data.

If only some public interfaces are analyzed (like memory.h), this takes 
of course
much less time. Keeping those public interfaces in one place (in 
directory 'include',
for example) supports this approach.

Regards,
Stefan Weil

On 12/15/2011 03:37 AM, Avi Kivity wrote:
> On 12/14/2011 06:20 PM, Anthony Liguori wrote:
>> By convention, documented headers now go in include/
>
> Dislike.

I've been planning on doing this for a while.  I think it's a useful way to help 
improve internal modularity.  It provides a consistent way to indicate which 
headers represent "public" internal interfaces (like the memory API) verses 
things that are really private headers specific to a submodule (say block_int.h).

We have a real problem internally with headers too.  It's almost surprising how 
many lack guards, don't have proper #includes, etc.  By moving all public 
headers to include/, it gives us a systematic way to go through, clean up 
various headers, and have an idea of how much work is left to be done.

>
>> +include $(SRC_PATH)/Makefile.docs
>> +
>>   $(common-obj-y): $(GENERATED_HEADERS)
>>   subdir-libcacard: $(oslib-obj-y) $(trace-obj-y) qemu-timer-common.o
>>
>> @@ -113,6 +115,8 @@ QEMU_CFLAGS+=$(CURL_CFLAGS)
>>
>>   QEMU_CFLAGS+=$(GLIB_CFLAGS)
>>
>> +QEMU_CFLAGS+=$(SRC_PATH)/include
>> +
>>   ui/cocoa.o: ui/cocoa.m
>
> Documentation should be built by default, so that errors in the format
> are detected (and break the build).

I agree, but since we now are dealing with a fork of a common tool, I didn't 
want to add a hard build dependency until I can get some feedback on whether 
upstream is willing to consider our patch.

>>
>> +
>> +gtkdoc: html/index.html
>> +
>> +html/index.html: $(DOC_SRC)
>> +	gtkdoc-scan --module=QEMU --source-dir=$(SRC_PATH)/include&&  \
>> +        cp $(SRC_PATH)/QEMU-docs.xml .&&  \
>> +	gtkdoc-mkdb --module=QEMU --output-format=xml --source-dir=$(SRC_PATH)/include&&  \
>> +	mkdir -p html&&  \
>> +        (cd html&&  gtkdoc-mkhtml QEMU ../QEMU-docs.xml&&  cd ..)&&  \
>> +	gtkdoc-fixxref --module=QEMU --module-dir=html
>> +
>
> Does this thing not support incremental builds?

As best as I can tell, no.  Every other tool I've looked as suffers from the 
same problem.

Regards,

Anthony Liguori

>> +++ b/QEMU-docs.xml
>> @@ -0,0 +1,31 @@
>> +<?xml version="1.0"?>
>> +<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
>> +               "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"
>> +[
>> +<!ENTITY % local.common.attrib "xmlns:xi  CDATA  #FIXED 'http://www.w3.org/2003/XInclude'">
>> +]>
>> +<book id="index">
>> +<bookinfo>
>> +<title>QEMU Reference Manual</title>
>> +<releaseinfo>
>> +      for QEMU 1.0.
>
> 1.1.
>

On 12/15/2011 03:30 PM, Anthony Liguori wrote:
> On 12/15/2011 03:37 AM, Avi Kivity wrote:
>> On 12/14/2011 06:20 PM, Anthony Liguori wrote:
>>> By convention, documented headers now go in include/
>>
>> Dislike.
>
> I've been planning on doing this for a while.  I think it's a useful
> way to help improve internal modularity.  It provides a consistent way
> to indicate which headers represent "public" internal interfaces (like
> the memory API) verses things that are really private headers specific
> to a submodule (say block_int.h).

If a submodule needs an internal header, it probably wants its own subdir.

>
> We have a real problem internally with headers too.  It's almost
> surprising how many lack guards, don't have proper #includes, etc.  By
> moving all public headers to include/, it gives us a systematic way to
> go through, clean up various headers, and have an idea of how much
> work is left to be done.

Still dislike.  But it's just dislike, not an opening shot into an
extended discussion that will expand into coding style, proposals for
changing the programming language, merging qemu into a subdirectory of
another project, etc, as much fun as it promises to be.  Do it if you
must, I'll live with it somehow.

>> Documentation should be built by default, so that errors in the format
>> are detected (and break the build).
>
> I agree, but since we now are dealing with a fork of a common tool, I
> didn't want to add a hard build dependency until I can get some
> feedback on whether upstream is willing to consider our patch.

Let's avoid a fork, either get it merged or find some other tool.

>> Does this thing not support incremental builds?
>
> As best as I can tell, no.  Every other tool I've looked as suffers
> from the same problem.

Yeah, it's equivalent to a compiler doing most of its work during the
link phase (=idea for patch).

On 12/15/2011 07:43 AM, Avi Kivity wrote:
> On 12/15/2011 03:30 PM, Anthony Liguori wrote:
>> On 12/15/2011 03:37 AM, Avi Kivity wrote:
>>> On 12/14/2011 06:20 PM, Anthony Liguori wrote:
>>>> By convention, documented headers now go in include/
>>>
>>> Dislike.
>>
>> I've been planning on doing this for a while.  I think it's a useful
>> way to help improve internal modularity.  It provides a consistent way
>> to indicate which headers represent "public" internal interfaces (like
>> the memory API) verses things that are really private headers specific
>> to a submodule (say block_int.h).
>
> If a submodule needs an internal header, it probably wants its own subdir.
>
>>
>> We have a real problem internally with headers too.  It's almost
>> surprising how many lack guards, don't have proper #includes, etc.  By
>> moving all public headers to include/, it gives us a systematic way to
>> go through, clean up various headers, and have an idea of how much
>> work is left to be done.
>
> Still dislike.  But it's just dislike, not an opening shot into an
> extended discussion that will expand into coding style, proposals for
> changing the programming language, merging qemu into a subdirectory of
> another project, etc, as much fun as it promises to be.  Do it if you
> must, I'll live with it somehow.

Man, it's just not easy anymore to start epic flame wars...  I guess I'll have 
to go back to coding.

>>> Documentation should be built by default, so that errors in the format
>>> are detected (and break the build).
>>
>> I agree, but since we now are dealing with a fork of a common tool, I
>> didn't want to add a hard build dependency until I can get some
>> feedback on whether upstream is willing to consider our patch.
>
> Let's avoid a fork, either get it merged or find some other tool.

This is definitely the best tool for the job.  If we're not willing to live with 
underscores and upstream won't carry the patch, forking is our only option.  We 
can do it as a git submodule though which will make life fairly easy.

Regards,

Anthony Liguori

>>> Does this thing not support incremental builds?
>>
>> As best as I can tell, no.  Every other tool I've looked as suffers
>> from the same problem.
>
> Yeah, it's equivalent to a compiler doing most of its work during the
> link phase (=idea for patch).
>

Am 15.12.2011 14:30, schrieb Anthony Liguori:
> On 12/15/2011 03:37 AM, Avi Kivity wrote:
>> On 12/14/2011 06:20 PM, Anthony Liguori wrote:
>>> By convention, documented headers now go in include/
>>
>> Dislike.

+1

> I've been planning on doing this for a while.  I think it's a useful way
> to help improve internal modularity.  It provides a consistent way to
> indicate which headers represent "public" internal interfaces (like the
> memory API) verses things that are really private headers specific to a
> submodule (say block_int.h).

So far you've always opposed the idea of a stable public API. If that's
still not what you want, then there's no real distinction here and an
include/ directory would give a wrong impression.

If you just want a TODO list, then surely having a shell script print
the *.h file names and copying them into the Wiki would be easier.

Andreas