GCOV: introduce --json-format.

Hi.

For some time we've been providing an intermediate format as
output of GCOV tool. It's documented in our code that primary
consumer of it is lcov. Apparently that's not true:
https://github.com/linux-test-project/lcov/issues/38#issuecomment-371203750

So that I decided to come up with providing the very similar intermediate
format in JSON format. It's much easier for consumers to work with.

I'm planning to leave the legacy format for GCC 9.1 and I'll document that
it's deprecated. We can then remove that in next release.

The patch contains a small change to json.{h,cc}, hope David can approve that?
Patch is tested on x86_64-linux-gnu.

Thanks,
Martin

gcc/ChangeLog:

2018-09-26  Martin Liska  <mliska@suse.cz>

	* Makefile.in: Make gcov dependant on json.o.
	* doc/gcov.texi: Document new option --json-format.
	* gcov.c (struct function_info): Rename name to m_name
	and demangled_name to m_demangled_name.
	(function_info::function_info): Likewise.
	(function_info::~function_info): Likewise.
	(function_info::get_demangled_name): New function.
	(function_info::get_name): Likewise.
	(main): Handle newly added flag_json_format.
	(print_usage): Likewise.
	(process_args): Likewise.
	(output_intermediate_line): Fix GNU coding style.
	(output_intermediate_json_line): New function.
	(get_gcov_intermediate_filename): Append .json for
	flag_json_format.
	(output_intermediate_file): Use get_name function.
	(output_json_intermediate_file): New.
	(generate_results): Use get_name function, output
	json intermediate format.
	(read_graph_file): Do not get demangled_name name.
	(read_count_file): Use get_name function.
	(solve_flow_graph): Likewise.
	(add_line_counts): Likewise.
	(accumulate_line_counts): Handle also flag_json_format.
	(output_function_details): Use get_name function.
	(output_lines): Likewise.
	* json.cc (test_writing_literals): Test new json::literal
	contructor.
	* json.h (class literal): Add new constructor for boolean
	type.
---
 gcc/Makefile.in   |   2 +-
 gcc/doc/gcov.texi |  12 ++-
 gcc/gcov.c        | 242 ++++++++++++++++++++++++++++++++++++----------
 gcc/json.cc       |   3 +
 gcc/json.h        |   3 +
 5 files changed, 211 insertions(+), 51 deletions(-)

On Thu, 2018-09-27 at 09:46 +0200, Martin Liška wrote:
> Hi.
> 
> For some time we've been providing an intermediate format as
> output of GCOV tool. It's documented in our code that primary
> consumer of it is lcov. Apparently that's not true:
> https://github.com/linux-test-project/lcov/issues/38#issuecomment-371
> 203750
> 
> So that I decided to come up with providing the very similar
> intermediate
> format in JSON format. It's much easier for consumers to work with.
> 
> I'm planning to leave the legacy format for GCC 9.1 and I'll document
> that
> it's deprecated. We can then remove that in next release.
> 
> The patch contains a small change to json.{h,cc}, hope David can
> approve that?
> Patch is tested on x86_64-linux-gnu.

I'm not officially a reviewer for the json stuff, but I can comment, at
least.  The changes to json.h/cc are fine by me, FWIW.

Some high-level observations:
* how big are the JSON files?  One of the comments at my Cauldron talk
on optimization records was a concern that the output could be very
large.  The JSON files compress really well, so maybe this patch should
gzip on output?  Though I don't know if it's appropriate for this case.
  iirc, gfortran's module code has an example of gzipping a
pretty_printer buffer.

* json::object::print doesn't preserve the insertion order of its
name/value pairs; they're written out in whatever order the hashing
leads to (maybe we should fix that though).  The top-level JSON value
in your file format is currently a JSON object containing "version"
metadata etc.  There's thus a chance that could be at the end, after
the data.  Perhaps the top-level JSON value should be an array instead
(as a "tuple"), to guarantee that the versioning metadata occurs at the
start?  Or are consumers assumed to read the whole file into memory and
traverse it, tree-like?

* Similar to the optimization records code, this patch is building a
tree of dynamically-allocated JSON objects in memory, and then printing
it to a pretty_printer, and flushing the pp's buffer to a FILE *.  This
is simple and flexible, but I suspect that I may need to rewrite this
for the optimization records case to avoid bloating the memory usage
(e.g. to eliminate the in-memory JSON tree in favor of printing as we
go). Would that concern apply here, or is the pattern OK?

* FWIW I'm working on DejaGnu test coverage for JSON output, but it's
ugly - a nasty shim between .exp and (optional) .py scripts for parsing
the JSON and verifying properties of it, sending the results back in a
form that DejaGnu can digest and integrate into the .sum/.log files.  I
hope to post it before stage 1 closes (I'd much prefer to have good
test coverage in place before optimizing how this stuff gets written)

[...snip...]
> diff --git a/gcc/gcov.c b/gcc/gcov.c
> index e255e4e3922..39d9329d6d0 100644
> --- a/gcc/gcov.c
> +++ b/gcc/gcov.c

[...snip...]

> @@ -1346,6 +1481,15 @@ generate_results (const char *file_name)
>  	}
>      }
>  
> +  json::object *root = new json::object ();

[...snip...]
>  
> -  if (flag_gcov_file && flag_intermediate_format && !flag_use_stdout)
> +  if (flag_gcov_file && flag_json_format)
> +    root->dump (out);
> +

It looks like "root" gets leaked here (and thus all of the objects in
the JSON tree also).   I don't know if that's a problem, but an easy
fix would be to make "root" be an on-stack object, rather than
allocated on the heap - this ought to lead everything to be cleaned up
when root's dtor runs.

Dave

On 9/27/18 3:55 PM, David Malcolm wrote:
> On Thu, 2018-09-27 at 09:46 +0200, Martin Liška wrote:
>> Hi.
>>
>> For some time we've been providing an intermediate format as
>> output of GCOV tool. It's documented in our code that primary
>> consumer of it is lcov. Apparently that's not true:
>> https://github.com/linux-test-project/lcov/issues/38#issuecomment-371
>> 203750
>>
>> So that I decided to come up with providing the very similar
>> intermediate
>> format in JSON format. It's much easier for consumers to work with.
>>
>> I'm planning to leave the legacy format for GCC 9.1 and I'll document
>> that
>> it's deprecated. We can then remove that in next release.
>>
>> The patch contains a small change to json.{h,cc}, hope David can
>> approve that?
>> Patch is tested on x86_64-linux-gnu.
> 
> I'm not officially a reviewer for the json stuff, but I can comment, at
> least.  The changes to json.h/cc are fine by me, FWIW.

Hello.

Appreciate your feedback!

> 
> Some high-level observations:
> * how big are the JSON files?  One of the comments at my Cauldron talk
> on optimization records was a concern that the output could be very
> large.  The JSON files compress really well, so maybe this patch should
> gzip on output?  Though I don't know if it's appropriate for this case.
>   iirc, gfortran's module code has an example of gzipping a
> pretty_printer buffer.

Probably sounds reasonable, for tramp3d I get: 5.8M, where original
intermediate format is 1.5M big. Gzipped JSON format is 216K.
I'll update the patch to address that.

> 
> * json::object::print doesn't preserve the insertion order of its
> name/value pairs; they're written out in whatever order the hashing
> leads to (maybe we should fix that though).  The top-level JSON value
> in your file format is currently a JSON object containing "version"
> metadata etc.  There's thus a chance that could be at the end, after
> the data.  Perhaps the top-level JSON value should be an array instead
> (as a "tuple"), to guarantee that the versioning metadata occurs at the
> start?  Or are consumers assumed to read the whole file into memory and
> traverse it, tree-like?

Well, it's not nice to do list from something that is object. It's micro-optimization
in my opinion.

On the other hand I would preserve the order of keys as they were added into
an object. Moreover, we can have option for that:

https://docs.python.org/3/library/json.html#basic-usage

Search for sort_keys. Similarly I would appreciate 'indent' option, it's handy
for debugging.

> 
> * Similar to the optimization records code, this patch is building a
> tree of dynamically-allocated JSON objects in memory, and then printing
> it to a pretty_printer, and flushing the pp's buffer to a FILE *.  This
> is simple and flexible, but I suspect that I may need to rewrite this
> for the optimization records case to avoid bloating the memory usage
> (e.g. to eliminate the in-memory JSON tree in favor of printing as we
> go). Would that concern apply here, or is the pattern OK?

One should not bloat memory in GCOV I guess.

> 
> * FWIW I'm working on DejaGnu test coverage for JSON output, but it's
> ugly - a nasty shim between .exp and (optional) .py scripts for parsing
> the JSON and verifying properties of it, sending the results back in a
> form that DejaGnu can digest and integrate into the .sum/.log files.  I
> hope to post it before stage 1 closes (I'd much prefer to have good
> test coverage in place before optimizing how this stuff gets written)

Would be good, then I would definitely add a test for GCOV JSON format.

> 
> [...snip...]
>> diff --git a/gcc/gcov.c b/gcc/gcov.c
>> index e255e4e3922..39d9329d6d0 100644
>> --- a/gcc/gcov.c
>> +++ b/gcc/gcov.c
> 
> [...snip...]
> 
>> @@ -1346,6 +1481,15 @@ generate_results (const char *file_name)
>>  	}
>>      }
>>  
>> +  json::object *root = new json::object ();
> 
> [...snip...]
>>  
>> -  if (flag_gcov_file && flag_intermediate_format && !flag_use_stdout)
>> +  if (flag_gcov_file && flag_json_format)
>> +    root->dump (out);
>> +
> 
> It looks like "root" gets leaked here (and thus all of the objects in
> the JSON tree also).   I don't know if that's a problem, but an easy
> fix would be to make "root" be an on-stack object, rather than
> allocated on the heap - this ought to lead everything to be cleaned up
> when root's dtor runs.

I've just tried valgrind and I can't see any leak in current implementation?

Martin

> 
> Dave
>

Hi.

I'm sending updated version of the patch. I made a research and it looks that
actually any significant consumer of GCOV does not use intermediate format:

https://github.com/gcovr/gcovr/issues/282
https://github.com/linux-test-project/lcov/issues/43
https://github.com/mozilla/grcov/issues/55

That said I made following changes:
- I removed current int. format and replaced that with JSON format
- documentation for the format is enhanced
- JSON files are generated by default gzipped

Patch survives gcov.exp tests.

Martin
From 94ea33a43e93aa76fdfe2b3b672589342296c2fb Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Wed, 26 Sep 2018 16:13:25 +0200
Subject: [PATCH] GCOV: introduce --json-format.

gcc/ChangeLog:

2018-10-10  Martin Liska  <mliska@suse.cz>

	* Makefile.in: Make dependency to json.o.
	* doc/gcov.texi: Document new JSON format, remove
	old intermediate format documentation.
	* gcov.c (struct function_info): Come up with m_name and
	m_demangled_name.
	(function_info::function_info): Initialize it.
	(function_info::~function_info): Release it.
	(main): Rename flag_intermediate_format to flag_json_format.
	(print_usage): Describe --json-format.
	(process_args): Set flag_json_format.
	(output_intermediate_line): Remove.
	(output_intermediate_json_line): Likewise.
	(get_gcov_intermediate_filename): Return new extension
	".gcov.json.gz".
	(output_intermediate_file): Implement JSON emission.
	(output_json_intermediate_file): Implement JSON emission.
	(generate_results): Use ::get_name for function name.
	Handle JSON output file.
	(read_graph_file): Use ::get_name instead of cplus_demangle.
	(read_count_file): Likewise.
	(solve_flow_graph): Likewise.
	(add_line_counts): Likewise.
	(accumulate_line_counts): Use new flag_json_format.
	(output_function_details): Use ::get_name instead of cplus_demangle.
	(output_lines): Likewise.
	* json.cc (test_writing_literals): Add new tests.
	* json.h (class literal): Add new boolean constructor.

gcc/testsuite/ChangeLog:

2018-10-10  Martin Liska  <mliska@suse.cz>

	* g++.dg/gcov/gcov-8.C: Do not check intermediate format.
	* lib/gcov.exp: Remove legacy verify-intermediate.
---
 gcc/Makefile.in                    |   7 +-
 gcc/doc/gcov.texi                  | 193 +++++++++++++--------
 gcc/gcov.c                         | 260 +++++++++++++++++------------
 gcc/json.cc                        |   3 +
 gcc/json.h                         |   3 +
 gcc/testsuite/g++.dg/gcov/gcov-8.C |   4 +-
 gcc/testsuite/lib/gcov.exp         |  55 ------
 7 files changed, 294 insertions(+), 231 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 116ed6ea8a5..908aa90c285 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -2910,10 +2910,13 @@ s-iov: build/gcov-iov$(build_exeext) $(BASEVER) $(DEVPHASE)
 	$(SHELL) $(srcdir)/../move-if-change tmp-gcov-iov.h gcov-iov.h
 	$(STAMP) s-iov
 
-GCOV_OBJS = gcov.o
+# gcov.o needs $(ZLIBINC) added to the include flags.
+CFLAGS-gcov.o += $(ZLIBINC)
+
+GCOV_OBJS = gcov.o json.o
 gcov$(exeext): $(GCOV_OBJS) $(LIBDEPS)
 	+$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) $(GCOV_OBJS) \
-		hash-table.o ggc-none.o $(LIBS) -o $@
+		hash-table.o ggc-none.o $(LIBS) $(ZLIB) -o $@
 GCOV_DUMP_OBJS = gcov-dump.o
 gcov-dump$(exeext): $(GCOV_DUMP_OBJS) $(LIBDEPS)
 	+$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) $(GCOV_DUMP_OBJS) \
diff --git a/gcc/doc/gcov.texi b/gcc/doc/gcov.texi
index 3b1b38aebfa..8046fd78e35 100644
--- a/gcc/doc/gcov.texi
+++ b/gcc/doc/gcov.texi
@@ -124,7 +124,7 @@ gcov [@option{-v}|@option{--version}] [@option{-h}|@option{--help}]
      [@option{-c}|@option{--branch-counts}]
      [@option{-d}|@option{--display-progress}]
      [@option{-f}|@option{--function-summaries}]
-     [@option{-i}|@option{--intermediate-format}]
+     [@option{-i}|@option{--json-format}]
      [@option{-j}|@option{--human-readable}]
      [@option{-k}|@option{--use-colors}]
      [@option{-l}|@option{--long-file-names}]
@@ -181,79 +181,142 @@ Display help about using @command{gcov} (on the standard output), and
 exit without doing any further processing.
 
 @item -i
-@itemx --intermediate-format
-Output gcov file in an easy-to-parse intermediate text format that can
-be used by @command{lcov} or other tools. The output is a single
-@file{.gcov} file per @file{.gcda} file. No source code is required.
+@itemx --json-format
+Output gcov file in an easy-to-parse JSON intermediate format
+which does not require source code for generation.  The JSON
+file is compressed with gzip compression algorithm
+and the files have @file{.gcov.json.gz} extension.
 
-The format of the intermediate @file{.gcov} file is plain text with
-one entry per line
+Structure of the JSON is following:
 
 @smallexample
-version:@var{gcc_version}
-cwd:@var{working_directory}
-file:@var{source_file_name}
-function:@var{start_line_number},@var{end_line_number},@var{execution_count},@var{function_name}
-lcount:@var{line number},@var{execution_count},@var{has_unexecuted_block}
-branch:@var{line_number},@var{branch_coverage_type}
-
-Where the @var{branch_coverage_type} is
-   notexec (Branch not executed)
-   taken (Branch executed and taken)
-   nottaken (Branch executed, but not taken)
+@{
+  "current_working_directory": @var{current_working_directory},
+  "format_version": @var{format_version},
+  "gcc_version": @var{gcc_version}
+  "files": [@var{file}]
+@}
 @end smallexample
 
-There can be multiple @var{file} entries in an intermediate gcov
-file. All entries following a @var{file} pertain to that source file
-until the next @var{file} entry.  If there are multiple functions that
-start on a single line, then corresponding lcount is repeated multiple
-times.
+Fields of the root element have following semantics:
 
-Here is a sample when @option{-i} is used in conjunction with @option{-b} option:
+@itemize @bullet
+@item
+@var{current_working_directory}: working directory where
+a compilation unit was compiled
+
+@item
+@var{format_version}: semantic version of the format
+
+@item
+@var{gcc_version}: version of the GCC compiler
+@end itemize
+
+Each @var{file} has the following form:
+
+@smallexample
+@{
+  "file": @var{file_name},
+  "functions": [@var{function}],
+  "lines": [@var{line}]
+@}
+@end smallexample
+
+Fields of the @var{file} element have following semantics:
+
+@itemize @bullet
+@item
+@var{file_name}: name of the source file
+@end itemize
+
+Each @var{function} has the following form:
+
+@smallexample
+@{
+  "blocks": @var{blocks},
+  "blocks_executed": @var{blocks_executed},
+  "demangled_name": "@var{demangled_name},
+  "end_line": @var{end_line},
+  "execution_count": @var{execution_count},
+  "name": @var{name},
+  "start_line": @var{start_line}
+@}
+@end smallexample
+
+Fields of the @var{function} element have following semantics:
+
+@itemize @bullet
+@item
+@var{blocks}: number of blocks that are in the function
+
+@item
+@var{blocks_executed}: number of executed blocks of the function
+
+@item
+@var{demangled_name}: demangled name of the function
+
+@item
+@var{end_line}: line in the source file where the function ends
+
+@item
+@var{execution_count}: number of executions of the function
+
+@item
+@var{name}: name of the function
+
+@item
+@var{start_line}: line in the source file where the function begins
+@end itemize
+
+Each @var{line} has the following form:
+
+@smallexample
+@{
+  "branches": [@var{branch}],
+  "count": @var{count},
+  "line_number": @var{line_number},
+  "unexecuted_block": @var{unexecuted_block}
+@}
+@end smallexample
+
+Branches are present only with @var{-b} option.
+Fields of the @var{line} element have following semantics:
+
+@itemize @bullet
+@item
+@var{count}: number of executions of the line
+
+@item
+@var{line_number}: line number
+
+@item
+@var{unexecuted_block}: flag whether the line contains an unexecuted block
+(not all statements on the line are executed)
+@end itemize
+
+Each @var{branch} has the following form:
 
 @smallexample
-version: 8.1.0 20180103
-cwd:/home/gcc/testcase
-file:tmp.cpp
-function:7,7,0,_ZN3FooIcEC2Ev
-function:7,7,1,_ZN3FooIiEC2Ev
-function:8,8,0,_ZN3FooIcE3incEv
-function:8,8,2,_ZN3FooIiE3incEv
-function:18,37,1,main
-lcount:7,0,1
-lcount:7,1,0
-lcount:8,0,1
-lcount:8,2,0
-lcount:18,1,0
-lcount:21,1,0
-branch:21,taken
-branch:21,nottaken
-lcount:23,1,0
-branch:23,taken
-branch:23,nottaken
-lcount:24,1,0
-branch:24,taken
-branch:24,nottaken
-lcount:25,1,0
-lcount:27,11,0
-branch:27,taken
-branch:27,taken
-lcount:28,10,0
-lcount:30,1,1
-branch:30,nottaken
-branch:30,taken
-lcount:32,1,0
-branch:32,nottaken
-branch:32,taken
-lcount:33,0,1
-branch:33,notexec
-branch:33,notexec
-lcount:35,1,0
-branch:35,taken
-branch:35,nottaken
-lcount:36,1,0
+@{
+  "count": @var{count},
+  "fallthrough": @var{fallthrough},
+  "throw": @var{throw}
+@}
 @end smallexample
 
+Fields of the @var{branch} element have following semantics:
+
+@itemize @bullet
+@item
+@var{count}: number of executions of the branch
+
+@item
+@var{fallthrough}: true when the branch is a fall through branch
+
+@item
+@var{throw}: true when the branch is an exceptional branch
+@end itemize
+
 @item -j
 @itemx --human-readable
 Write counts in human readable format (like 24.6k).
@@ -842,7 +905,7 @@ some summary information.
 
 It is not recommended to access the coverage files directly.
 Consumers should use the intermediate format that is provided
-by @command{gcov} tool via @option{--intermediate-format} option.
+by @command{gcov} tool via @option{--json-format} option.
 
 @node Cross-profiling
 @section Data File Relocation to Support Cross-Profiling
diff --git a/gcc/gcov.c b/gcc/gcov.c
index e255e4e3922..26d352ce7f1 100644
--- a/gcc/gcov.c
+++ b/gcc/gcov.c
@@ -44,7 +44,10 @@ along with Gcov; see the file COPYING3.  If not see
 #include "version.h"
 #include "demangle.h"
 #include "color-macros.h"
+#include "pretty-print.h"
+#include "json.h"
 
+#include <zlib.h>
 #include <getopt.h>
 
 #include "md5.h"
@@ -221,6 +224,10 @@ line_info::has_block (block_info *needle)
   return std::find (blocks.begin (), blocks.end (), needle) != blocks.end ();
 }
 
+/* Output demangled function names.  */
+
+static int flag_demangled_names = 0;
+
 /* Describes a single function. Contains an array of basic blocks.  */
 
 struct function_info
@@ -241,8 +248,8 @@ struct function_info
   }
 
   /* Name of function.  */
-  char *name;
-  char *demangled_name;
+  char *m_name;
+  char *m_demangled_name;
   unsigned ident;
   unsigned lineno_checksum;
   unsigned cfg_checksum;
@@ -284,6 +291,32 @@ struct function_info
 
   /* Next function.  */
   struct function_info *next;
+
+  /*  Get demangled name of a function.  The demangled name
+      is converted when it is used for the first time.  */
+  char *get_demangled_name ()
+  {
+    if (m_demangled_name == NULL)
+      {
+	m_demangled_name = cplus_demangle (m_name, DMGL_PARAMS);
+	if (!m_demangled_name)
+	  m_demangled_name = m_name;
+      }
+
+    return m_demangled_name;
+  }
+
+  /* Get name of the function based on flag_demangled_names.  */
+  char *get_name ()
+  {
+    return flag_demangled_names ? get_demangled_name () : m_name;
+  }
+
+  /* Return number of basic blocks (without entry and exit block).  */
+  unsigned get_block_count ()
+  {
+    return blocks.size () - 2;
+  }
 };
 
 /* Function info comparer that will sort functions according to starting
@@ -477,13 +510,9 @@ static int flag_use_stdout = 0;
 
 static int flag_display_progress = 0;
 
-/* Output *.gcov file in intermediate format used by 'lcov'.  */
+/* Output *.gcov file in JSON intermediate format used by consumers.  */
 
-static int flag_intermediate_format = 0;
-
-/* Output demangled function names.  */
-
-static int flag_demangled_names = 0;
+static int flag_json_format = 0;
 
 /* For included files, make the gcov output file name include the name
    of the input source file.  For example, if x.h is included in a.c,
@@ -576,7 +605,7 @@ static char *mangle_name (const char *, char *);
 static void release_structures (void);
 extern int main (int, char **);
 
-function_info::function_info (): name (NULL), demangled_name (NULL),
+function_info::function_info (): m_name (NULL), m_demangled_name (NULL),
   ident (0), lineno_checksum (0), cfg_checksum (0), has_catch (0),
   artificial (0), is_group (0),
   blocks (), blocks_executed (0), counts (),
@@ -596,9 +625,9 @@ function_info::~function_info ()
 	  free (arc);
 	}
     }
-  if (flag_demangled_names && demangled_name != name)
-    free (demangled_name);
-  free (name);
+  if (m_demangled_name != m_name)
+    free (m_demangled_name);
+  free (m_name);
 }
 
 bool function_info::group_line_p (unsigned n, unsigned src_idx)
@@ -807,7 +836,7 @@ main (int argc, char **argv)
 		argc - first_arg);
       process_file (argv[argno]);
 
-      if (flag_intermediate_format || argno == argc - 1)
+      if (flag_json_format || argno == argc - 1)
 	{
 	  process_all_functions ();
 	  generate_results (argv[argno]);
@@ -836,7 +865,7 @@ print_usage (int error_p)
   fnotice (file, "  -d, --display-progress          Display progress information\n");
   fnotice (file, "  -f, --function-summaries        Output summaries for each function\n");
   fnotice (file, "  -h, --help                      Print this help, then exit\n");
-  fnotice (file, "  -i, --intermediate-format       Output .gcov file in intermediate text format\n");
+  fnotice (file, "  -i, --json-format		    Output JSON intermediate format into .gcov.json.gz file\n");
   fnotice (file, "  -j, --human-readable            Output human readable numbers\n");
   fnotice (file, "  -k, --use-colors                Emit colored output\n");
   fnotice (file, "  -l, --long-file-names           Use long output file names for included\n\
@@ -881,7 +910,7 @@ static const struct option options[] =
   { "all-blocks",           no_argument,       NULL, 'a' },
   { "branch-probabilities", no_argument,       NULL, 'b' },
   { "branch-counts",        no_argument,       NULL, 'c' },
-  { "intermediate-format",  no_argument,       NULL, 'i' },
+  { "json-format",	    no_argument,       NULL, 'i' },
   { "human-readable",	    no_argument,       NULL, 'j' },
   { "no-output",            no_argument,       NULL, 'n' },
   { "long-file-names",      no_argument,       NULL, 'l' },
@@ -963,12 +992,12 @@ process_args (int argc, char **argv)
 	  flag_unconditional = 1;
 	  break;
 	case 'i':
-          flag_intermediate_format = 1;
-          flag_gcov_file = 1;
-          break;
-        case 'd':
-          flag_display_progress = 1;
-          break;
+	  flag_json_format = 1;
+	  flag_gcov_file = 1;
+	  break;
+	case 'd':
+	  flag_display_progress = 1;
+	  break;
 	case 'x':
 	  flag_hash_filenames = 1;
 	  break;
@@ -990,17 +1019,23 @@ process_args (int argc, char **argv)
   return optind;
 }
 
-/* Output intermediate LINE sitting on LINE_NUM to output file F.  */
+/* Output intermediate LINE sitting on LINE_NUM to JSON OBJECT.  */
 
 static void
-output_intermediate_line (FILE *f, line_info *line, unsigned line_num)
+output_intermediate_json_line (json::array *object,
+			       line_info *line, unsigned line_num)
 {
   if (!line->exists)
     return;
 
-  fprintf (f, "lcount:%u,%s,%d\n", line_num,
-	   format_gcov (line->count, 0, -1),
-	   line->has_unexecuted_block);
+  json::object *lineo = new json::object ();
+  lineo->set ("line_number", new json::number (line_num));
+  lineo->set ("count", new json::number (line->count));
+  lineo->set ("unexecuted_block",
+	      new json::literal (line->has_unexecuted_block));
+
+  json::array *branches = new json::array ();
+  lineo->set ("branches", branches);
 
   vector<arc_info *>::const_iterator it;
   if (flag_branches)
@@ -1009,21 +1044,16 @@ output_intermediate_line (FILE *f, line_info *line, unsigned line_num)
       {
 	if (!(*it)->is_unconditional && !(*it)->is_call_non_return)
 	  {
-	    const char *branch_type;
-	    /* branch:<line_num>,<branch_coverage_infoype>
-	       branch_coverage_infoype
-	       : notexec (Branch not executed)
-	       : taken (Branch executed and taken)
-	       : nottaken (Branch executed, but not taken)
-	       */
-	    if ((*it)->src->count)
-		 branch_type
-			= ((*it)->count > 0) ? "taken" : "nottaken";
-	    else
-	      branch_type = "notexec";
-	    fprintf (f, "branch:%d,%s\n", line_num, branch_type);
+	    json::object *branch = new json::object ();
+	    branch->set ("count", new json::number ((*it)->count));
+	    branch->set ("throw", new json::literal ((*it)->is_throw));
+	    branch->set ("fallthrough",
+			 new json::literal ((*it)->fall_through));
+	    branches->append (branch);
 	  }
       }
+
+  object->append (lineo);
 }
 
 /* Get the name of the gcov file.  The return value must be free'd.
@@ -1038,7 +1068,7 @@ output_intermediate_line (FILE *f, line_info *line, unsigned line_num)
 static char *
 get_gcov_intermediate_filename (const char *file_name)
 {
-  const char *gcov = ".gcov";
+  const char *gcov = ".gcov.json.gz";
   char *result;
   const char *cptr;
 
@@ -1051,34 +1081,44 @@ get_gcov_intermediate_filename (const char *file_name)
   return result;
 }
 
-/* Output the result in intermediate format used by 'lcov'.
-
-The intermediate format contains a single file named 'foo.cc.gcov',
-with no source code included.
-
-The default gcov outputs multiple files: 'foo.cc.gcov',
-'iostream.gcov', 'ios_base.h.gcov', etc. with source code
-included. Instead the intermediate format here outputs only a single
-file 'foo.cc.gcov' similar to the above example. */
+/* Output the result in JSON intermediate format.
+   Source info SRC is dumped into JSON_FILES which is JSON array.  */
 
 static void
-output_intermediate_file (FILE *gcov_file, source_info *src)
+output_json_intermediate_file (json::array *json_files, source_info *src)
 {
-  fprintf (gcov_file, "version:%s\n", version_string);
-  fprintf (gcov_file, "file:%s\n", src->name);    /* source file name */
-  fprintf (gcov_file, "cwd:%s\n", bbg_cwd);
+  json::object *root = new json::object ();
+  json_files->append (root);
+
+  root->set ("file", new json::string (src->name));
+
+  json::array *functions = new json::array ();
+  root->set ("functions", functions);
 
   std::sort (src->functions.begin (), src->functions.end (),
 	     function_line_start_cmp ());
   for (vector<function_info *>::iterator it = src->functions.begin ();
        it != src->functions.end (); it++)
     {
-      /* function:<name>,<line_number>,<execution_count> */
-      fprintf (gcov_file, "function:%d,%d,%s,%s\n", (*it)->start_line,
-	       (*it)->end_line, format_gcov ((*it)->blocks[0].count, 0, -1),
-	       flag_demangled_names ? (*it)->demangled_name : (*it)->name);
+      json::object *function = new json::object ();
+      function->set ("name", new json::string ((*it)->m_name));
+      function->set ("demangled_name",
+		     new json::string ((*it)->get_demangled_name ()));
+      function->set ("start_line", new json::number ((*it)->start_line));
+      function->set ("end_line", new json::number ((*it)->end_line));
+      function->set ("blocks",
+		     new json::number ((*it)->get_block_count ()));
+      function->set ("blocks_executed",
+		     new json::number ((*it)->blocks_executed));
+      function->set ("execution_count",
+		     new json::number ((*it)->blocks[0].count));
+
+      functions->append (function);
     }
 
+  json::array *lineso = new json::array ();
+  root->set ("lines", lineso);
+
   for (unsigned line_num = 1; line_num <= src->lines.size (); line_num++)
     {
       vector<function_info *> fns = src->get_functions_at_location (line_num);
@@ -1091,13 +1131,13 @@ output_intermediate_file (FILE *gcov_file, source_info *src)
 	  for (unsigned i = 0; i < lines.size (); i++)
 	    {
 	      line_info *line = &lines[i];
-	      output_intermediate_line (gcov_file, line, line_num + i);
+	      output_intermediate_json_line (lineso, line, line_num + i);
 	    }
 	}
 
       /* Follow with lines associated with the source file.  */
       if (line_num < src->lines.size ())
-	output_intermediate_line (gcov_file, &src->lines[line_num], line_num);
+	output_intermediate_json_line (lineso, &src->lines[line_num], line_num);
     }
 }
 
@@ -1301,8 +1341,7 @@ output_gcov_file (const char *file_name, source_info *src)
 static void
 generate_results (const char *file_name)
 {
-  FILE *gcov_intermediate_file = NULL;
-  char *gcov_intermediate_filename = NULL;
+  char *gcov_intermediate_filename;
 
   for (vector<function_info *>::iterator it = functions.begin ();
        it != functions.end (); it++)
@@ -1311,7 +1350,7 @@ generate_results (const char *file_name)
       coverage_info coverage;
 
       memset (&coverage, 0, sizeof (coverage));
-      coverage.name = flag_demangled_names ? fn->demangled_name : fn->name;
+      coverage.name = fn->get_name ();
       add_line_counts (flag_function_summary ? &coverage : NULL, fn);
       if (flag_function_summary)
 	{
@@ -1333,18 +1372,15 @@ generate_results (const char *file_name)
 	file_name = canonicalize_name (file_name);
     }
 
-  if (flag_gcov_file && flag_intermediate_format && !flag_use_stdout)
-    {
-      /* Open the intermediate file.  */
-      gcov_intermediate_filename = get_gcov_intermediate_filename (file_name);
-      gcov_intermediate_file = fopen (gcov_intermediate_filename, "w");
-      if (!gcov_intermediate_file)
-	{
-	  fnotice (stderr, "Cannot open intermediate output file %s\n",
-		   gcov_intermediate_filename);
-	  return;
-	}
-    }
+  gcov_intermediate_filename = get_gcov_intermediate_filename (file_name);
+
+  json::object *root = new json::object ();
+  root->set ("format_version", new json::string ("1"));
+  root->set ("gcc_version", new json::string (version_string));
+  root->set ("current_working_directory", new json::string (bbg_cwd));
+
+  json::array *json_files = new json::array ();
+  root->set ("files", json_files);
 
   for (vector<source_info>::iterator it = sources.begin ();
        it != sources.end (); it++)
@@ -1372,11 +1408,8 @@ generate_results (const char *file_name)
       total_executed += src->coverage.lines_executed;
       if (flag_gcov_file)
 	{
-	  if (flag_intermediate_format)
-	    /* Output the intermediate format without requiring source
-	       files.  This outputs a section to a *single* file.  */
-	    output_intermediate_file ((flag_use_stdout
-				       ? stdout : gcov_intermediate_file), src);
+	  if (flag_json_format)
+	    output_json_intermediate_file (json_files, src);
 	  else
 	    {
 	      if (flag_use_stdout)
@@ -1393,11 +1426,35 @@ generate_results (const char *file_name)
 	}
     }
 
-  if (flag_gcov_file && flag_intermediate_format && !flag_use_stdout)
+  if (flag_gcov_file && flag_json_format)
     {
-      /* Now we've finished writing the intermediate file.  */
-      fclose (gcov_intermediate_file);
-      XDELETEVEC (gcov_intermediate_filename);
+      if (flag_use_stdout)
+	{
+	  root->dump (stdout);
+	  printf ("\n");
+	}
+      else
+	{
+	  pretty_printer pp;
+	  root->print (&pp);
+	  pp_formatted_text (&pp);
+
+	  gzFile output = gzopen (gcov_intermediate_filename, "w");
+	  if (output == NULL)
+	    {
+	      fnotice (stderr, "Cannot open JSON output file %s\n",
+		       gcov_intermediate_filename);
+	      return;
+	    }
+
+	  if (gzputs (output, pp_formatted_text (&pp)) == EOF
+	      || gzclose (output))
+	    {
+	      fnotice (stderr, "Error writing JSON output file %s\n",
+		       gcov_intermediate_filename);
+	      return;
+	    }
+	}
     }
 
   if (!file_name)
@@ -1634,13 +1691,7 @@ read_graph_file (void)
 
 	  fn = new function_info ();
 	  functions.push_back (fn);
-	  fn->name = function_name;
-	  if (flag_demangled_names)
-	    {
-	      fn->demangled_name = cplus_demangle (fn->name, DMGL_PARAMS);
-	      if (!fn->demangled_name)
-		fn->demangled_name = fn->name;
-	    }
+	  fn->m_name = function_name;
 	  fn->ident = ident;
 	  fn->lineno_checksum = lineno_checksum;
 	  fn->cfg_checksum = cfg_checksum;
@@ -1656,7 +1707,7 @@ read_graph_file (void)
 	{
 	  if (!fn->blocks.empty ())
 	    fnotice (stderr, "%s:already seen blocks for '%s'\n",
-		     bbg_file_name, fn->name);
+		     bbg_file_name, fn->get_name ());
 	  else
 	    fn->blocks.resize (gcov_read_unsigned ());
 	}
@@ -1862,7 +1913,7 @@ read_count_file (void)
 	    {
 	    mismatch:;
 	      fnotice (stderr, "%s:profile mismatch for '%s'\n",
-		       da_file_name, fn->name);
+		       da_file_name, fn->get_name ());
 	      goto cleanup;
 	    }
 	}
@@ -1927,12 +1978,12 @@ solve_flow_graph (function_info *fn)
 
   if (fn->blocks.size () < 2)
     fnotice (stderr, "%s:'%s' lacks entry and/or exit blocks\n",
-	     bbg_file_name, fn->name);
+	     bbg_file_name, fn->get_name ());
   else
     {
       if (fn->blocks[ENTRY_BLOCK].num_pred)
 	fnotice (stderr, "%s:'%s' has arcs to entry block\n",
-		 bbg_file_name, fn->name);
+		 bbg_file_name, fn->get_name ());
       else
 	/* We can't deduce the entry block counts from the lack of
 	   predecessors.  */
@@ -1940,7 +1991,7 @@ solve_flow_graph (function_info *fn)
 
       if (fn->blocks[EXIT_BLOCK].num_succ)
 	fnotice (stderr, "%s:'%s' has arcs from exit block\n",
-		 bbg_file_name, fn->name);
+		 bbg_file_name, fn->get_name ());
       else
 	/* Likewise, we can't deduce exit block counts from the lack
 	   of its successors.  */
@@ -2149,7 +2200,7 @@ solve_flow_graph (function_info *fn)
     if (!fn->blocks[i].count_valid)
       {
 	fnotice (stderr, "%s:graph is unsolvable for '%s'\n",
-		 bbg_file_name, fn->name);
+		 bbg_file_name, fn->get_name ());
 	break;
       }
 }
@@ -2553,7 +2604,8 @@ add_line_counts (coverage_info *coverage, function_info *fn)
     }
 
   if (!has_any_line)
-    fnotice (stderr, "%s:no lines for '%s'\n", bbg_file_name, fn->name);
+    fnotice (stderr, "%s:no lines for '%s'\n", bbg_file_name,
+	     fn->get_name ());
 }
 
 /* Accumulate info for LINE that belongs to SRC source file.  If ADD_COVERAGE
@@ -2633,7 +2685,7 @@ accumulate_line_counts (source_info *src)
 
   /* If not using intermediate mode, sum lines of group functions and
      add them to lines that live in a source file.  */
-  if (!flag_intermediate_format)
+  if (!flag_json_format)
     for (vector<function_info *>::iterator it = src->functions.begin ();
 	 it != src->functions.end (); it++)
       {
@@ -2895,7 +2947,7 @@ output_line_details (FILE *f, const line_info *line, unsigned line_num)
 /* Output detail statistics about function FN to file F.  */
 
 static void
-output_function_details (FILE *f, const function_info *fn)
+output_function_details (FILE *f, function_info *fn)
 {
   if (!flag_branches)
     return;
@@ -2908,15 +2960,13 @@ output_function_details (FILE *f, const function_info *fn)
     if (arc->fake)
       return_count -= arc->count;
 
-  fprintf (f, "function %s",
-	   flag_demangled_names ? fn->demangled_name : fn->name);
+  fprintf (f, "function %s", fn->get_name ());
   fprintf (f, " called %s",
 	   format_gcov (called_count, 0, -1));
   fprintf (f, " returned %s",
 	   format_gcov (return_count, called_count, 0));
   fprintf (f, " blocks executed %s",
-	   format_gcov (fn->blocks_executed, fn->blocks.size () - 2,
-			0));
+	   format_gcov (fn->blocks_executed, fn->get_block_count (), 0));
   fprintf (f, "\n");
 }
 
@@ -3028,9 +3078,7 @@ output_lines (FILE *gcov_file, const source_info *src)
 
 	      fprintf (gcov_file, FN_SEPARATOR);
 
-	      string fn_name
-		= flag_demangled_names ? fn->demangled_name : fn->name;
-
+	      string fn_name = fn->get_name ();
 	      if (flag_use_colors)
 		{
 		  fn_name.insert (0, SGR_SEQ (COLOR_FG_CYAN));
diff --git a/gcc/json.cc b/gcc/json.cc
index 3ead98073f6..a0c439560e9 100644
--- a/gcc/json.cc
+++ b/gcc/json.cc
@@ -296,6 +296,9 @@ test_writing_literals ()
   assert_print_eq (literal (JSON_TRUE), "true");
   assert_print_eq (literal (JSON_FALSE), "false");
   assert_print_eq (literal (JSON_NULL), "null");
+
+  assert_print_eq (literal (true), "true");
+  assert_print_eq (literal (false), "false");
 }
 
 /* Run all of the selftests within this file.  */
diff --git a/gcc/json.h b/gcc/json.h
index 154d9e1b575..e99141e71e1 100644
--- a/gcc/json.h
+++ b/gcc/json.h
@@ -154,6 +154,9 @@ class literal : public value
  public:
   literal (enum kind kind) : m_kind (kind) {}
 
+  /* Construct literal for a boolean value.  */
+  literal (bool value): m_kind (value ? JSON_TRUE : JSON_FALSE) {}
+
   enum kind get_kind () const FINAL OVERRIDE { return m_kind; }
   void print (pretty_printer *pp) const FINAL OVERRIDE;
 
diff --git a/gcc/testsuite/g++.dg/gcov/gcov-8.C b/gcc/testsuite/g++.dg/gcov/gcov-8.C
index 7acab4672ff..1c97c35d8c1 100644
--- a/gcc/testsuite/g++.dg/gcov/gcov-8.C
+++ b/gcc/testsuite/g++.dg/gcov/gcov-8.C
@@ -1,5 +1,3 @@
-/* Verify that intermediate coverage format can be generated for simple code. */
-
 /* { dg-options "-fprofile-arcs -ftest-coverage" } */
 /* { dg-do run { target native } } */
 
@@ -32,4 +30,4 @@ int main()
   foo();
 }
 
-/* { dg-final { run-gcov intermediate { -i -b gcov-8.C } } } */
+/* { dg-final { run-gcov { -b gcov-8.C } } } */
diff --git a/gcc/testsuite/lib/gcov.exp b/gcc/testsuite/lib/gcov.exp
index f35ca59fbe3..a7b8c0a1ef4 100644
--- a/gcc/testsuite/lib/gcov.exp
+++ b/gcc/testsuite/lib/gcov.exp
@@ -83,61 +83,6 @@ proc verify-lines { testname testcase file } {
 }
 
 
-#
-# verify-intermediate -- check that intermediate file has certain lines
-#
-# TESTNAME is the name of the test, including unique flags.
-# TESTCASE is the name of the test.
-# FILE is the name of the gcov output file.
-#
-# Checks are very loose, they are based on certain tags being present
-# in the output. They do not check for exact expected execution
-# counts. For that the regular gcov format should be checked.
-#
-proc verify-intermediate { testname testcase file } {
-    set failed 0
-    set srcfile 0
-    set function 0
-    set lcount 0
-    set branch 0
-    set fd [open $file r]
-    while { [gets $fd line] >= 0 } {
-	if [regexp "^file:" $line] {
-	    incr srcfile
-	}
-	if [regexp "^function:(\[0-9\]+),(\[0-9\]+),.*" $line] {
-	    incr function
-	}
-	if [regexp "^lcount:(\[0-9\]+),(\[0-9\]+),(\[01\])" $line] {
-	    incr lcount
-	}
-	if [regexp "^branch:(\[0-9\]+),(taken|nottaken|notexec)" $line] {
-	    incr branch
-	}
-    }
-
-    # We should see at least one tag of each type
-    if {$srcfile == 0} {
-	fail "$testname expected 'file:' tag not found"
-	incr failed
-    }
-    if {$function == 0} {
-	fail "$testname expected 'function:' tag not found"
-	incr failed
-    }
-    if {$lcount == 0} {
-	fail "$testname expected 'lcount:' tag not found"
-	incr failed
-    }
-    if {$branch == 0} {
-	fail "$testname expected 'branch:' tag not found"
-	incr failed
-    }
-    close $fd
-    return $failed
-}
-
-
 #
 # verify-branches -- check that branch percentages are as expected
 #

On 10/10/18 1:46 PM, Martin Liška wrote:
> Hi.
> 
> I'm sending updated version of the patch. I made a research and it looks that
> actually any significant consumer of GCOV does not use intermediate format:
> 
> https://github.com/gcovr/gcovr/issues/282
> https://github.com/linux-test-project/lcov/issues/43
> https://github.com/mozilla/grcov/issues/55
> 
> That said I made following changes:
> - I removed current int. format and replaced that with JSON format
> - documentation for the format is enhanced
> - JSON files are generated by default gzipped
> 
> Patch survives gcov.exp tests.
> 
> Martin
> 

I've just installed the patch as r265587.

Martin