Patchwork [2/2] tests: add gcov target

login
register
mail settings
Submitter Blue Swirl
Date June 10, 2012, 1:04 p.m.
Message ID <dbeda171a924975032f9f69d3d7dc91fe067f1de.1339333471.git.blauwirbel@gmail.com>
Download mbox | patch
Permalink /patch/163997/
State New
Headers show

Comments

Blue Swirl - June 10, 2012, 1:04 p.m.
Add support for compiling for GCOV test coverage, enabled
with '--enable-gcov' during configure.

After tests, test coverage can be reported with 'make gcov'.

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
---
 configure      |   19 ++++++++++++++++++-
 tests/Makefile |   40 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 1 deletions(-)
Paolo Bonzini - June 10, 2012, 4:04 p.m.
Il 10/06/2012 15:04, Blue Swirl ha scritto:
> @@ -2851,10 +2859,14 @@ fi
>  # End of CC checks
>  # After here, no more $cc or $ld runs
>  
> -if test "$debug" = "no" ; then
> +if test "$gcov" = "yes" ; then
> +  CFLAGS="-fprofile-arcs -ftest-coverage -g $CFLAGS"
> +  LDFLAGS="-fprofile-arcs -ftest-coverage $LDFLAGS"
> +elif test "$debug" = "no" ; then
>    CFLAGS="-O2 $CFLAGS"
>  fi

Why does need to be mutually exclusive?

>  
> +
>  # Consult white-list to determine whether to enable werror
>  # by default.  Only enable by default for git builds
>  z_version=`cut -f3 -d. $source_path/VERSION`
> @@ -3033,6 +3045,8 @@ echo "OpenGL support    $opengl"
>  echo "libiscsi support  $libiscsi"
>  echo "build guest agent $guest_agent"
>  echo "coroutine backend $coroutine_backend"
> +echo "gcov              $gcov_tool"
> +echo "gcov enabled      $gcov"
>  
>  if test "$sdl_too_old" = "yes"; then
>  echo "-> Your SDL version is too old - please upgrade to have SDL support"
> @@ -3441,6 +3455,9 @@ echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
>  echo "EXESUF=$EXESUF" >> $config_host_mak
>  echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
>  echo "POD2MAN=$POD2MAN" >> $config_host_mak
> +if test "$gcov" = "yes" ; then
> +  echo "GCOV=$gcov_tool" >> $config_host_mak
> +fi
>  
>  # generate list of library paths for linker script
>  
> diff --git a/tests/Makefile b/tests/Makefile
> index 2e754c3..d265daa 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -1,18 +1,42 @@
>  export SRC_PATH
>  
>  check-unit-y = tests/check-qdict$(EXESUF)
> +GCOV_FILES = $(SRC_PATH)/qdict.c

Ouch, ugly.  Why is it needed?

Paolo
Andreas Färber - June 10, 2012, 4:08 p.m.
Am 10.06.2012 15:04, schrieb Blue Swirl:
> Add support for compiling for GCOV test coverage, enabled
> with '--enable-gcov' during configure.
> 
> After tests, test coverage can be reported with 'make gcov'.
> 
> Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
> ---
>  configure      |   19 ++++++++++++++++++-
>  tests/Makefile |   40 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 58 insertions(+), 1 deletions(-)
> 
> diff --git a/configure b/configure
> index b4fb715..d773f4e 100755
> --- a/configure
> +++ b/configure
[...]
> @@ -2851,10 +2859,14 @@ fi
>  # End of CC checks
>  # After here, no more $cc or $ld runs
>  
> -if test "$debug" = "no" ; then
> +if test "$gcov" = "yes" ; then
> +  CFLAGS="-fprofile-arcs -ftest-coverage -g $CFLAGS"
> +  LDFLAGS="-fprofile-arcs -ftest-coverage $LDFLAGS"
> +elif test "$debug" = "no" ; then
>    CFLAGS="-O2 $CFLAGS"
>  fi

Is it intentional that no -O option is being specified for gcov?

>  
> +
>  # Consult white-list to determine whether to enable werror
>  # by default.  Only enable by default for git builds
>  z_version=`cut -f3 -d. $source_path/VERSION`
[snip]

Am I understanding correctly that this gcov setup must be done in a
separate build dir from an actually "usable" build?

Small improvement: Please either include a cover letter for a threaded
series or send unrelated patches separately. Groups replies more nicely.

Regards,
Andreas
Blue Swirl - June 10, 2012, 6:07 p.m.
On Sun, Jun 10, 2012 at 4:04 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 10/06/2012 15:04, Blue Swirl ha scritto:
>> @@ -2851,10 +2859,14 @@ fi
>>  # End of CC checks
>>  # After here, no more $cc or $ld runs
>>
>> -if test "$debug" = "no" ; then
>> +if test "$gcov" = "yes" ; then
>> +  CFLAGS="-fprofile-arcs -ftest-coverage -g $CFLAGS"
>> +  LDFLAGS="-fprofile-arcs -ftest-coverage $LDFLAGS"
>> +elif test "$debug" = "no" ; then
>>    CFLAGS="-O2 $CFLAGS"
>>  fi
>
> Why does need to be mutually exclusive?

Optimization may skip generating some code so coverage would not be
recorded for these areas. Recommended by GNU:
http://gcc.gnu.org/onlinedocs/gcc/Gcov-and-Optimization.html#Gcov-and-Optimization

>
>>
>> +
>>  # Consult white-list to determine whether to enable werror
>>  # by default.  Only enable by default for git builds
>>  z_version=`cut -f3 -d. $source_path/VERSION`
>> @@ -3033,6 +3045,8 @@ echo "OpenGL support    $opengl"
>>  echo "libiscsi support  $libiscsi"
>>  echo "build guest agent $guest_agent"
>>  echo "coroutine backend $coroutine_backend"
>> +echo "gcov              $gcov_tool"
>> +echo "gcov enabled      $gcov"
>>
>>  if test "$sdl_too_old" = "yes"; then
>>  echo "-> Your SDL version is too old - please upgrade to have SDL support"
>> @@ -3441,6 +3455,9 @@ echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
>>  echo "EXESUF=$EXESUF" >> $config_host_mak
>>  echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
>>  echo "POD2MAN=$POD2MAN" >> $config_host_mak
>> +if test "$gcov" = "yes" ; then
>> +  echo "GCOV=$gcov_tool" >> $config_host_mak
>> +fi
>>
>>  # generate list of library paths for linker script
>>
>> diff --git a/tests/Makefile b/tests/Makefile
>> index 2e754c3..d265daa 100644
>> --- a/tests/Makefile
>> +++ b/tests/Makefile
>> @@ -1,18 +1,42 @@
>>  export SRC_PATH
>>
>>  check-unit-y = tests/check-qdict$(EXESUF)
>> +GCOV_FILES = $(SRC_PATH)/qdict.c
>
> Ouch, ugly.  Why is it needed?

Gcov needs to know both the source file and the object. Probably it
can't get this information from the object or .gc{da,no} files.

>
> Paolo
Blue Swirl - June 10, 2012, 6:14 p.m.
On Sun, Jun 10, 2012 at 4:08 PM, Andreas Färber <afaerber@suse.de> wrote:
> Am 10.06.2012 15:04, schrieb Blue Swirl:
>> Add support for compiling for GCOV test coverage, enabled
>> with '--enable-gcov' during configure.
>>
>> After tests, test coverage can be reported with 'make gcov'.
>>
>> Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
>> ---
>>  configure      |   19 ++++++++++++++++++-
>>  tests/Makefile |   40 ++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 58 insertions(+), 1 deletions(-)
>>
>> diff --git a/configure b/configure
>> index b4fb715..d773f4e 100755
>> --- a/configure
>> +++ b/configure
> [...]
>> @@ -2851,10 +2859,14 @@ fi
>>  # End of CC checks
>>  # After here, no more $cc or $ld runs
>>
>> -if test "$debug" = "no" ; then
>> +if test "$gcov" = "yes" ; then
>> +  CFLAGS="-fprofile-arcs -ftest-coverage -g $CFLAGS"
>> +  LDFLAGS="-fprofile-arcs -ftest-coverage $LDFLAGS"
>> +elif test "$debug" = "no" ; then
>>    CFLAGS="-O2 $CFLAGS"
>>  fi
>
> Is it intentional that no -O option is being specified for gcov?

Yes, see my reply to Paolo.

>
>>
>> +
>>  # Consult white-list to determine whether to enable werror
>>  # by default.  Only enable by default for git builds
>>  z_version=`cut -f3 -d. $source_path/VERSION`
> [snip]
>
> Am I understanding correctly that this gcov setup must be done in a
> separate build dir from an actually "usable" build?

Well, running the program would attempt to sprinkle the build
directory with .gc{da,no} files.

>
> Small improvement: Please either include a cover letter for a threaded
> series or send unrelated patches separately. Groups replies more nicely.

I did not want to add 50% extra messages or invent the BLURB.

>
> Regards,
> Andreas
>
> --
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Paolo Bonzini - June 11, 2012, 4:56 a.m.
Il 10/06/2012 20:07, Blue Swirl ha scritto:
> > >  check-unit-y = tests/check-qdict$(EXESUF)
> > > +GCOV_FILES = $(SRC_PATH)/qdict.c
> >
> > Ouch, ugly.  Why is it needed?
> 
> Gcov needs to know both the source file and the object. Probably it
> can't get this information from the object or .gc{da,no} files.

Yes, I meant why in QEMU.  In the new buildsystem all .o files match the
original hierarchy in the source tree, so it should be possible to avoid
this and get it from substitutions on the names of the .gcda files.

Paolo

Patch

diff --git a/configure b/configure
index b4fb715..d773f4e 100755
--- a/configure
+++ b/configure
@@ -153,6 +153,8 @@  strip_opt="yes"
 tcg_interpreter="no"
 bigendian="no"
 mingw32="no"
+gcov="no"
+gcov_tool="gcov"
 EXESUF=""
 prefix="/usr/local"
 mandir="\${prefix}/share/man"
@@ -557,6 +559,8 @@  for opt do
   ;;
   --python=*) python="$optarg"
   ;;
+  --gcov=*) gcov_tool="$optarg"
+  ;;
   --smbd=*) smbd="$optarg"
   ;;
   --extra-cflags=*)
@@ -577,6 +581,8 @@  for opt do
   ;;
   --enable-gprof) gprof="yes"
   ;;
+  --enable-gcov) gcov="yes"
+  ;;
   --static)
     static="yes"
     LDFLAGS="-static $LDFLAGS"
@@ -1111,6 +1117,8 @@  echo "  --disable-guest-agent    disable building of the QEMU Guest Agent"
 echo "  --enable-guest-agent     enable building of the QEMU Guest Agent"
 echo "  --with-coroutine=BACKEND coroutine backend. Supported options:"
 echo "                           gthread, ucontext, sigaltstack, windows"
+echo "  --enable-gcov            enable test coverage analysis with gcov"
+echo "  --gcov=GCOV              use specified gcov [$gcov_tool]"
 echo ""
 echo "NOTE: The object files are built at the place where configure is launched"
 exit 1
@@ -2851,10 +2859,14 @@  fi
 # End of CC checks
 # After here, no more $cc or $ld runs
 
-if test "$debug" = "no" ; then
+if test "$gcov" = "yes" ; then
+  CFLAGS="-fprofile-arcs -ftest-coverage -g $CFLAGS"
+  LDFLAGS="-fprofile-arcs -ftest-coverage $LDFLAGS"
+elif test "$debug" = "no" ; then
   CFLAGS="-O2 $CFLAGS"
 fi
 
+
 # Consult white-list to determine whether to enable werror
 # by default.  Only enable by default for git builds
 z_version=`cut -f3 -d. $source_path/VERSION`
@@ -3033,6 +3045,8 @@  echo "OpenGL support    $opengl"
 echo "libiscsi support  $libiscsi"
 echo "build guest agent $guest_agent"
 echo "coroutine backend $coroutine_backend"
+echo "gcov              $gcov_tool"
+echo "gcov enabled      $gcov"
 
 if test "$sdl_too_old" = "yes"; then
 echo "-> Your SDL version is too old - please upgrade to have SDL support"
@@ -3441,6 +3455,9 @@  echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
 echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
+if test "$gcov" = "yes" ; then
+  echo "GCOV=$gcov_tool" >> $config_host_mak
+fi
 
 # generate list of library paths for linker script
 
diff --git a/tests/Makefile b/tests/Makefile
index 2e754c3..d265daa 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,18 +1,42 @@ 
 export SRC_PATH
 
 check-unit-y = tests/check-qdict$(EXESUF)
+GCOV_FILES = $(SRC_PATH)/qdict.c
 check-unit-y += tests/check-qfloat$(EXESUF)
+GCOV_FILES += $(SRC_PATH)/qfloat.c
 check-unit-y += tests/check-qint$(EXESUF)
+GCOV_FILES += $(SRC_PATH)/qint.c
 check-unit-y += tests/check-qstring$(EXESUF)
+GCOV_FILES += $(SRC_PATH)/qstring.c
 check-unit-y += tests/check-qlist$(EXESUF)
+GCOV_FILES += $(SRC_PATH)/qlist.c
 check-unit-y += tests/check-qjson$(EXESUF)
+GCOV_FILES += $(SRC_PATH)/qjson.c
 check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
+GCOV_QAPI_FILES += $(SRC_PATH)/qapi/qmp-output-visitor.c
 check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
+GCOV_QAPI_FILES += $(SRC_PATH)/qapi/qmp-input-visitor.c
 check-unit-y += tests/test-qmp-input-strict$(EXESUF)
 check-unit-y += tests/test-qmp-commands$(EXESUF)
+GCOV_QAPI_FILES += $(SRC_PATH)/qapi/qmp-dispatch.c
 check-unit-y += tests/test-string-input-visitor$(EXESUF)
+GCOV_QAPI_FILES += $(SRC_PATH)/qapi/string-input-visitor.c
 check-unit-y += tests/test-string-output-visitor$(EXESUF)
+GCOV_QAPI_FILES += $(SRC_PATH)/qapi/string-output-visitor.c
 check-unit-y += tests/test-coroutine$(EXESUF)
+ifeq ($(CONFIG_WIN32),y)
+GCOV_FILES += $(SRC_PATH)/coroutine-win32.c
+else
+ifeq ($(CONFIG_UCONTEXT_COROUTINE),y)
+GCOV_FILES += $(SRC_PATH)/coroutine-ucontext.c
+else
+ifeq ($(CONFIG_SIGALTSTACK_COROUTINE),y)
+GCOV_FILES += $(SRC_PATH)/coroutine-sigaltstack.c
+else
+GCOV_FILES += $(SRC_PATH)/coroutine-gthread.c
+endif
+endif
+endif
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -21,8 +45,10 @@  check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 check-qtest-i386-y = tests/fdc-test$(EXESUF)
 check-qtest-i386-y += tests/rtc-test$(EXESUF)
 check-qtest-x86_64-y = $(check-qtest-i386-y)
+GCOV_I386_SOFTMMU_FILES += $(SRC_PATH)/hw/mc146818rtc.c
 check-qtest-sparc-y = tests/m48t59-test$(EXESUF)
 check-qtest-sparc64-y = tests/m48t59-test$(EXESUF)
+GCOV_LIBHW64_FILES += $(SRC_PATH)/hw/m48t59.c
 
 GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
 
@@ -88,6 +114,8 @@  check-help:
 	@echo " make check-unit           Run qobject tests"
 	@echo " make check-block          Run block tests"
 	@echo " make check-report.html    Generates an HTML test report"
+	@echo " make gcov                 Report test coverage"
+	@echo " make gcov-zerocounters    Zero test coverage counters"
 	@echo
 	@echo "Please note that HTML reports do not regenerate if the unit tests"
 	@echo "has not changed."
@@ -95,6 +123,8 @@  check-help:
 	@echo "The variable SPEED can be set to control the gtester speed setting."
 	@echo "Default options are -k and (for make V=1) --verbose; they can be"
 	@echo "changed with variable GTESTER_OPTIONS."
+	@echo
+	@echo "For test coverage, please run 'make gcov-zerocounters check gcov'."
 
 SPEED = quick
 GTESTER_OPTIONS = -k $(if $(V),--verbose,-q)
@@ -127,6 +157,16 @@  check-report.xml: $(patsubst %,check-report-qtest-%.xml, $(QTEST_TARGETS)) check
 check-report.html: check-report.xml
 	$(call quiet-command,gtester-report $< > $@, "  GEN    $@")
 
+# Gcov test coverage
+.PHONY: gcov gcov-zerocounters
+gcov:
+	@$(GCOV) -n $(GCOV_FILES)
+	@(cd i386-softmmu/hw && $(GCOV) -n $(GCOV_I386_SOFTMMU_FILES))
+	@(cd libhw64/hw && $(GCOV) -n $(GCOV_LIBHW64_FILES))
+	@(cd qapi && $(GCOV) -n $(GCOV_QAPI_FILES))
+
+gcov-zerocounters:
+	rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
 
 # Other tests