From patchwork Fri May 13 01:03:36 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rong Xu X-Patchwork-Id: 95409 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 8C871B6EFE for ; Fri, 13 May 2011 11:04:12 +1000 (EST) Received: (qmail 28369 invoked by alias); 13 May 2011 01:04:09 -0000 Received: (qmail 28322 invoked by uid 22791); 13 May 2011 01:04:01 -0000 X-SWARE-Spam-Status: No, hits=-0.8 required=5.0 tests=AWL, BAYES_20, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, SPF_HELO_PASS, TW_CF, TW_CP, TW_GD, TW_VB, TW_VF, TW_VP, T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from smtp-out.google.com (HELO smtp-out.google.com) (216.239.44.51) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 13 May 2011 01:03:41 +0000 Received: from hpaq7.eem.corp.google.com (hpaq7.eem.corp.google.com [172.25.149.7]) by smtp-out.google.com with ESMTP id p4D13da2018833; Thu, 12 May 2011 18:03:40 -0700 Received: from rong.mtv.corp.google.com (rong.mtv.corp.google.com [172.18.116.40]) by hpaq7.eem.corp.google.com with ESMTP id p4D13bPB032745; Thu, 12 May 2011 18:03:37 -0700 Received: by rong.mtv.corp.google.com (Postfix, from userid 104659) id E567FC2972; Thu, 12 May 2011 18:03:36 -0700 (PDT) To: reply@codereview.appspotmail.com, gcc-patches@gcc.gnu.org Subject: [google] support for building Linux kernel with FDO (issue4523061) Message-Id: <20110513010336.E567FC2972@rong.mtv.corp.google.com> Date: Thu, 12 May 2011 18:03:36 -0700 (PDT) From: xur@google.com (Rong Xu) X-System-Of-Record: true Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org This patch add support to build Linux kernel with FDO. Building Linux kernel with FDO needs both compiler change and kernel changes. This part only contains the gcc changes. I'll attach kernel changes to the reveiew thread for reference. Source files gcov-io.c gcov-io.h and libgcov.c will be copied to kernel source and directly used in the kernel build. When building for kernel, we assume option '-D__KERNEL__' is used. To enable profile generation, we need to a.1) set the following config: CONFIG_GCOV_KERNEL=y CONFIG_GCOV_PROFILE_ALL=y a.2) set COMPILER_LIB_PATH to the absolute path to the parent directory of "gcov-src". "gcov-src" is a directory containing the following 4 files: gcov-iov.h gcov-io.h gcov-io.c and libgcov.c. It usually sits at ${GCC_ROOT}/lib/gcc/x86_64-unknown-linux-gnu/${GCC_VERSION}/ a.3) set CFLAGS_GCOV as "-fprofile-generate" and build the kernel (prof_gen). To obtain the profile data b.1) install profile_gen kernel, mount debugfs. b.2) reset the profile count: echo 1 > /sys/kernel/debug/gcov/reset b.3) run the benchmark b.4) cp -r /sys/kernel/debug/gcov// To build FDO optimized kernel: c.1 ) same as a.1). c.2 ) same as a.2). c.3) set set CFLAGS_GCOV as "-fprofile-use -fprofile-correction -Wcoverage-mismatch" and build. Performance results: Tested with 2.6.34 and 2.6.36 kernel with stand-alone benchmarks: tbench(~23%), dbench(~4%), unixbench(1%-2%), no noticable regressions. This patch has been testd with bootstraps, regression test, standalong kernel benchmarks (tbench, dbench, membench, unixbench, iozone, specjbb, and kernbench), and google internal FDO tests. 2011-05-12 Rong Xu * gcc/gcov-io.c (revision 173717): FDO support to build Linux kernel. * gcc/gcov-io.h (revision 173717): FDO support to build Linux kernel. * gcc/coverage.c (revision 173717): set a flag if building for Linux kernel. * gcc/tree-profile.c (revision 173717): don't emit TLS declarations for Linux kernel builds. * gcc/libgcov.c (revision 173717): FDO support to build Linux kernel. --- This patch is available for review at http://codereview.appspot.com/4523061 Index: gcc/gcov-io.c =================================================================== --- gcc/gcov-io.c (revision 173717) +++ gcc/gcov-io.c (working copy) @@ -32,6 +32,10 @@ static void gcov_allocate (unsigned); #endif +#ifdef __GCOV_KERNEL__ +struct gcov_var gcov_var ATTRIBUTE_HIDDEN; +#endif + static inline gcov_unsigned_t from_file (gcov_unsigned_t value) { #if !IN_LIBGCOV @@ -54,6 +58,7 @@ Return zero on failure, >0 on opening an existing file and <0 on creating a new one. */ +#ifndef __GCOV_KERNEL__ GCOV_LINKAGE int #if IN_LIBGCOV gcov_open (const char *name) @@ -148,7 +153,24 @@ return 1; } +#else /* __GCOV_KERNEL__ */ +extern _GCOV_FILE *gcov_current_file; + +GCOV_LINKAGE int +gcov_open (const char *name) +{ + gcov_var.start = 0; + gcov_var.offset = gcov_var.length = 0; + gcov_var.overread = -1u; + gcov_var.error = 0; + gcov_var.file = gcov_current_file; + gcov_var.mode = 1; + + return 1; +} +#endif /* __GCOV_KERNEL__ */ + /* Close the current gcov file. Flushes data to disk. Returns nonzero on failure or error flag set. */ @@ -161,7 +183,7 @@ if (gcov_var.offset && gcov_var.mode < 0) gcov_write_block (gcov_var.offset); #endif - fclose (gcov_var.file); + _GCOV_fclose (gcov_var.file); gcov_var.file = 0; gcov_var.length = 0; } @@ -217,7 +239,7 @@ static void gcov_write_block (unsigned size) { - if (fwrite (gcov_var.buffer, size << 2, 1, gcov_var.file) != 1) + if (_GCOV_fwrite (gcov_var.buffer, size << 2, 1, gcov_var.file) != 1) gcov_var.error = 1; gcov_var.start += size; gcov_var.offset -= size; @@ -413,7 +435,7 @@ gcov_allocate (gcov_var.length + words); excess = gcov_var.alloc - gcov_var.length; #endif - excess = fread (gcov_var.buffer + gcov_var.length, + excess = _GCOV_fread (gcov_var.buffer + gcov_var.length, 1, excess << 2, gcov_var.file) >> 2; gcov_var.length += excess; if (gcov_var.length < words) @@ -554,6 +576,10 @@ GCOV_LINKAGE void gcov_sync (gcov_position_t base, gcov_unsigned_t length) { +#ifdef __GCOV_KERNEL__ + /* should not reach this point */ + gcc_assert (0); +#else /* __GCOV_KERNEL__ */ gcc_assert (gcov_var.mode > 0); base += length; if (base - gcov_var.start <= gcov_var.length) @@ -561,9 +587,10 @@ else { gcov_var.offset = gcov_var.length = 0; - fseek (gcov_var.file, base << 2, SEEK_SET); - gcov_var.start = ftell (gcov_var.file) >> 2; + _GCOV_fseek (gcov_var.file, base << 2, SEEK_SET); + gcov_var.start = _GCOV_ftell (gcov_var.file) >> 2; } +#endif /* __GCOV_KERNEL__ */ } #endif @@ -576,8 +603,8 @@ gcc_assert (gcov_var.mode < 0); if (gcov_var.offset) gcov_write_block (gcov_var.offset); - fseek (gcov_var.file, base << 2, SEEK_SET); - gcov_var.start = ftell (gcov_var.file) >> 2; + _GCOV_fseek (gcov_var.file, base << 2, SEEK_SET); + gcov_var.start = _GCOV_ftell (gcov_var.file) >> 2; } /* Truncate the gcov file at the current position. */ @@ -585,6 +612,10 @@ GCOV_LINKAGE void gcov_truncate (void) { +#ifdef __GCOV_KERNEL__ + /* should not reach this point */ + gcc_assert (0); +#else /* __GCOV_KERNEL__ */ long offs; int filenum; gcc_assert (gcov_var.mode < 0); @@ -594,6 +625,7 @@ filenum = fileno (gcov_var.file); if (offs == -1 || filenum == -1 || ftruncate (filenum, offs)) gcov_var.error = 1; +#endif /* __GCOV_KERNEL__ */ } #endif @@ -611,3 +643,105 @@ return status.st_mtime; } #endif /* IN_GCOV */ + +#ifdef __GCOV_KERNEL__ + +/* File fclose operation in kernel mode. */ + +int +kernel_file_fclose (gcov_kernel_vfile *fp) +{ + return 0; +} + +/* File ftell operation in kernel mode. It currently should not + be called. */ + +long +kernel_file_ftell (gcov_kernel_vfile *fp) +{ + gcc_assert (0); /* should not reach here */ + return 0; +} + +/* File fseek operation in kernel mode. It should only be called + with OFFSET==0 and WHENCE==0 to a freshly opened file. */ + +int +kernel_file_fseek (gcov_kernel_vfile *fp, long offset, int whence) +{ + gcc_assert (offset == 0 && whence == 0 && fp->count == 0); + return 0; +} + +/* File ftruncate operation in kernel mode. It currently should not + be called. */ + +int +kernel_file_ftruncate (gcov_kernel_vfile *fp, off_t value) +{ + gcc_assert (0); /* should not reach here */ + return 0; +} + +/* File fread operation in kernel mode. It currently should not + be called. */ + +int +kernel_file_fread (void *ptr, size_t size, size_t nitems, + gcov_kernel_vfile *fp) +{ + gcc_assert (0); /* should not reach here */ + return 0; +} + +/* File fwrite operation in kernel mode. It outputs the data + to a buffer in the virual file. */ + +int +kernel_file_fwrite (const void *ptr, size_t size, + size_t nitems, gcov_kernel_vfile *fp) +{ + char *vbuf; + unsigned vsize, vpos; + unsigned len; + + if (!fp) return 0; + + vbuf = fp->buf; + vsize = fp->size; + vpos = fp->count; + + if (vsize <= vpos) + { + printk (KERN_ERR + "GCOV_KERNEL: something wrong: vbuf=%p vsize=%u vpos=%u\n", + vbuf, vsize, vpos); + return 0; + } + len = vsize - vpos; + len /= size; + + if (len > nitems) + len = nitems; + + memcpy (vbuf+vpos, ptr, size*len); + fp->count += len*size; + + if (len != nitems) + printk (KERN_ERR + "GCOV_KERNEL: something wrong: size=%lu nitems=%lu ret=%d\n", + size, nitems, len); + return len; +} + +/* File fileno operation in kernel mode. It currently should not + be called. */ + +int +kernel_file_fileno (gcov_kernel_vfile *fp) +{ + gcc_assert (0); /* should not reach here */ + return 0; +} +#endif /* GCOV_KERNEL */ Index: gcc/gcov-io.h =================================================================== --- gcc/gcov-io.h (revision 173717) +++ gcc/gcov-io.h (working copy) @@ -163,6 +163,88 @@ #ifndef GCC_GCOV_IO_H #define GCC_GCOV_IO_H +#ifdef __KERNEL__ +#ifndef __GCOV_KERNEL__ +#define __GCOV_KERNEL__ +#endif /* __GCOV_KERNEL__ */ +#endif /* __KERNEL__ */ + +#ifdef __GCOV_KERNEL__ +#define GCOV_LINKAGE /* nothing */ + +/* We need the definitions for + BITS_PER_UNIT and + LONG_LONG_TYPE_SIZE + They are defined in gcc/defaults.h and gcc/config/ + (like, gcc/config/i386/i386.h). And it can be overridden by setting + in build scripts. Here I hardcoded the value for x86. + Todo: using a program to auto-generate the vaules in build time. */ +#define BITS_PER_UNIT 8 +#define LONG_LONG_TYPE_SIZE 64 + +/* There are many gcc_assertions. Set the vaule to 1 if we want a warning + message if the assertion fails. */ +#ifndef ENABLE_ASSERT_CHECKING +#define ENABLE_ASSERT_CHECKING 1 +#endif + +#include +#endif /* __GCOV_KERNEL__ */ + +/* Wrappers to the file operations. */ +#ifndef __GCOV_KERNEL__ +# define _GCOV_FILE FILE +# define _GCOV_fclose fclose +# define _GCOV_ftell ftell +# define _GCOV_fseek fseek +# define _GCOV_ftruncate ftruncate +# define _GCOV_fread fread +# define _GCOV_fwrite fwrite +# define _GCOV_fread fread +# define _GCOV_fileno fileno +#else /* __GCOV_KERNEL__ */ +/* In Linux kernel mode, a virtual file is used for file operations. */ +struct gcov_info; +typedef struct { + long size; /* size of buf */ + long count; /* element written into buf */ + struct gcov_info *info; + char buf[0]; +} gcov_kernel_vfile; + +# define _GCOV_FILE gcov_kernel_vfile + +/* gcc_assert() prints out a warning if the check fails. It + will not abort. */ +#if ENABLE_ASSERT_CHECKING +# define gcc_assert(EXPR) \ + ((void)(!(EXPR) ? printk (KERN_WARNING \ + "GCOV assertion fails: func=%s line=%d\n", \ + __FUNCTION__, __LINE__), 0 : 0)) +#else +# define gcc_assert(EXPR) ((void)(0 && (EXPR))) +#endif + +/* Wrappers to the file operations. */ +# define _GCOV_fclose kernel_file_fclose +# define _GCOV_ftell kernel_file_ftell +# define _GCOV_fseek kernel_file_fseek +# define _GCOV_ftruncate kernel_file_ftruncate +# define _GCOV_fread kernel_file_fread +# define _GCOV_fwrite kernel_file_fwrite +# define _GCOV_fileno kernel_file_fileno + +/* Declarations for virtual files operations. */ +extern int kernel_file_fclose (gcov_kernel_vfile *); +extern long kernel_file_ftell (gcov_kernel_vfile *); +extern int kernel_file_fseek (gcov_kernel_vfile *, long, int); +extern int kernel_file_ftruncate (gcov_kernel_vfile *, off_t); +extern int kernel_file_fread (void *, size_t, size_t, + gcov_kernel_vfile *); +extern int kernel_file_fwrite (const void *, size_t, size_t, + gcov_kernel_vfile *); +extern int kernel_file_fileno(gcov_kernel_vfile *); +#endif /* GCOV_KERNEL */ #if IN_LIBGCOV #undef FUNC_ID_WIDTH @@ -264,7 +346,9 @@ is not also used in a DSO. */ #if IN_LIBGCOV +#ifndef __GCOV_KERNEL__ #include "tconfig.h" +#endif /* __GCOV_KERNEL__ */ #define gcov_var __gcov_var #define gcov_open __gcov_open @@ -609,9 +693,9 @@ /* Optimum number of gcov_unsigned_t's read from or written to disk. */ #define GCOV_BLOCK_SIZE (1 << 10) -GCOV_LINKAGE struct gcov_var +struct gcov_var { - FILE *file; + _GCOV_FILE *file; gcov_position_t start; /* Position of first byte of block */ unsigned offset; /* Read/write position within the block. */ unsigned length; /* Read limit in the block. */ @@ -631,8 +715,16 @@ size_t alloc; gcov_unsigned_t *buffer; #endif -} gcov_var ATTRIBUTE_HIDDEN; +}; +/* In kernel mode, move gcov_var definition to gcov-io.c + to avoid dulipcate definitions. */ +#ifndef __GCOV_KERNEL__ +GCOV_LINKAGE struct gcov_var gcov_var ATTRIBUTE_HIDDEN; +#else +extern struct gcov_var gcov_var; +#endif + /* Functions for reading and writing gcov files. In libgcov you can open the file for reading then writing. Elsewhere you can open the file either for reading or for writing. When reading a file you may @@ -679,6 +771,7 @@ static void gcov_rewrite (void); GCOV_LINKAGE void gcov_seek (gcov_position_t /*position*/) ATTRIBUTE_HIDDEN; GCOV_LINKAGE void gcov_truncate (void) ATTRIBUTE_HIDDEN; +GCOV_LINKAGE unsigned gcov_gcda_file_size (struct gcov_info *); #else /* Available outside libgcov */ GCOV_LINKAGE const char *gcov_read_string (void); @@ -729,7 +822,7 @@ gcov_var.mode = -1; gcov_var.start = 0; gcov_var.offset = 0; - fseek (gcov_var.file, 0L, SEEK_SET); + _GCOV_fseek (gcov_var.file, 0L, SEEK_SET); } #endif Index: gcc/coverage.c =================================================================== --- gcc/coverage.c (revision 173717) +++ gcc/coverage.c (working copy) @@ -1949,6 +1949,10 @@ *tail = e; } +extern bool is_kernel_build; + +#define KERNEL_BUILD_PREDEF_STRING "__KERNEL__" + /* Copies the macro def or undef CPP_DEF and saves the copy in a list. IS_DEF is a flag indicating if CPP_DEF represents a -D or -U. */ @@ -1961,6 +1965,11 @@ strcpy (s + 1, cpp_def); str_list_append (&cpp_defines_head, &cpp_defines_tail, s); num_cpp_defines++; + + /* When -D__KERNEL__ is in the option list, we assume this is + compilation for Linux Kernel. */ + if (!strcmp(cpp_def, KERNEL_BUILD_PREDEF_STRING)) + is_kernel_build = is_def; } /* Copies the -imacro/-include FILENAME and saves the copy in a list. */ Index: gcc/tree-profile.c =================================================================== --- gcc/tree-profile.c (revision 173717) +++ gcc/tree-profile.c (working copy) @@ -76,6 +76,10 @@ static GTY(()) tree ptr_void; static GTY(()) tree gcov_info_decl; +/* When -D__KERNEL__ is in the option list, we assume this is a + compilation for Linux Kernel. */ +bool is_kernel_build; + /* Do initialization work for the edge profiler. */ /* Add code: @@ -102,7 +106,7 @@ ptr_void); TREE_PUBLIC (ic_void_ptr_var) = 1; DECL_EXTERNAL (ic_void_ptr_var) = 1; - if (targetm.have_tls) + if (targetm.have_tls && !is_kernel_build) DECL_TLS_MODEL (ic_void_ptr_var) = decl_default_tls_model (ic_void_ptr_var); @@ -113,7 +117,7 @@ gcov_type_ptr); TREE_PUBLIC (ic_gcov_type_ptr_var) = 1; DECL_EXTERNAL (ic_gcov_type_ptr_var) = 1; - if (targetm.have_tls) + if (targetm.have_tls && !is_kernel_build) DECL_TLS_MODEL (ic_gcov_type_ptr_var) = decl_default_tls_model (ic_gcov_type_ptr_var); } @@ -126,7 +130,7 @@ TREE_STATIC (ic_void_ptr_var) = 1; TREE_PUBLIC (ic_void_ptr_var) = 0; DECL_INITIAL (ic_void_ptr_var) = NULL; - if (targetm.have_tls) + if (targetm.have_tls && !is_kernel_build) DECL_TLS_MODEL (ic_void_ptr_var) = decl_default_tls_model (ic_void_ptr_var); @@ -138,7 +142,7 @@ TREE_STATIC (ic_gcov_type_ptr_var) = 1; TREE_PUBLIC (ic_gcov_type_ptr_var) = 0; DECL_INITIAL (ic_gcov_type_ptr_var) = NULL; - if (targetm.have_tls) + if (targetm.have_tls && !is_kernel_build) DECL_TLS_MODEL (ic_gcov_type_ptr_var) = decl_default_tls_model (ic_gcov_type_ptr_var); } @@ -318,7 +322,7 @@ TREE_PUBLIC (gcov_sample_counter_decl) = 1; DECL_EXTERNAL (gcov_sample_counter_decl) = 1; DECL_ARTIFICIAL (gcov_sample_counter_decl) = 1; - if (targetm.have_tls) + if (targetm.have_tls && !is_kernel_build) DECL_TLS_MODEL (gcov_sample_counter_decl) = decl_default_tls_model (gcov_sample_counter_decl); assemble_variable (gcov_sample_counter_decl, 0, 0, 0); @@ -1479,8 +1483,9 @@ build_pointer_type (gcov_type_node)); DECL_ARTIFICIAL (dc_gcov_type_ptr_var) = 1; DECL_EXTERNAL (dc_gcov_type_ptr_var) = 1; - DECL_TLS_MODEL (dc_gcov_type_ptr_var) = - decl_default_tls_model (dc_gcov_type_ptr_var); + if (!is_kernel_build) + DECL_TLS_MODEL (dc_gcov_type_ptr_var) = + decl_default_tls_model (dc_gcov_type_ptr_var); dc_void_ptr_var = build_decl (UNKNOWN_LOCATION, VAR_DECL, @@ -1488,8 +1493,9 @@ ptr_void); DECL_ARTIFICIAL (dc_void_ptr_var) = 1; DECL_EXTERNAL (dc_void_ptr_var) = 1; - DECL_TLS_MODEL (dc_void_ptr_var) = - decl_default_tls_model (dc_void_ptr_var); + if (!is_kernel_build) + DECL_TLS_MODEL (dc_void_ptr_var) = + decl_default_tls_model (dc_void_ptr_var); } add_referenced_var (gcov_info_decl); Index: gcc/libgcov.c =================================================================== --- gcc/libgcov.c (revision 173717) +++ gcc/libgcov.c (working copy) @@ -25,10 +25,29 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see . */ +/* Assume compiling for Linux Kernel if __KERNEL__ is defined. */ +#ifdef __KERNEL__ + /* Define MACROs to be used by kernel compilation. */ +# define L_gcov +# define L_gcov_interval_profiler +# define L_gcov_pow2_profiler +# define L_gcov_one_value_profiler +# define L_gcov_indirect_call_profiler +# define L_gcov_average_profiler +# define L_gcov_ior_profiler + +# define TARGET_VTABLE_USES_DESCRIPTORS 0 +# define HAVE_CC_TLS 0 +# define __GCOV_KERNEL__ + +# define IN_LIBGCOV 1 +# define IN_GCOV 0 +#else /* __KERNEL__ */ #include "tconfig.h" #include "tsystem.h" #include "coretypes.h" #include "tm.h" +#endif /* __KERNEL__ */ #if 1 #define THREAD_PREFIX __thread @@ -36,6 +55,7 @@ #define THREAD_PREFIX #endif +#ifndef __GCOV_KERNEL__ #if defined(inhibit_libc) #define IN_LIBGCOV (-1) #else @@ -46,6 +66,8 @@ #define GCOV_LINKAGE /* nothing */ #endif #endif +#endif /* __GCOV_KERNEL__ */ + #include "gcov-io.h" #if defined(inhibit_libc) @@ -73,16 +95,35 @@ #else +#ifndef __GCOV_KERNEL__ #include #if GCOV_LOCKED #include #include #include #endif +#endif /* __GCOV_KERNEL__ */ #ifdef L_gcov #include "gcov-io.c" +/* Utility function for outputing errors. */ +static int +gcov_error (const char *fmt, ...) +{ + int ret; + va_list argp; + va_start (argp, fmt); +#ifdef __GCOV_KERNEL__ + ret = vprintk (fmt, argp); +#else + ret = vfprintf (stderr, fmt, argp); +#endif + va_end (argp); + return ret; +} + +#ifndef __GCOV_KERNEL__ /* Sampling rate. */ extern gcov_unsigned_t __gcov_sampling_rate; static int gcov_sampling_rate_initialized = 0; @@ -100,6 +141,10 @@ /* Chain of per-object gcov structures. */ extern struct gcov_info *__gcov_list; +/* Size of the longest file name. */ +static size_t gcov_max_filename = 0; +#endif /* __GCOV_KERNEL__ */ + /* Unique identifier assigned to each module (object file). */ static gcov_unsigned_t gcov_cur_module_id = 0; @@ -121,13 +166,45 @@ object file included in multiple programs. */ static gcov_unsigned_t gcov_crc32; -/* Size of the longest file name. */ -static size_t gcov_max_filename = 0; - /* Dynamic call graph build and form module groups. */ void __gcov_compute_module_groups (void) ATTRIBUTE_HIDDEN; void __gcov_finalize_dyn_callgraph (void) ATTRIBUTE_HIDDEN; +/* Profile summary for the gdca file, used in sanity check? */ +static struct gcov_summary all; + +/* Profile summary for this program in current exeuction. */ +static struct gcov_summary this_program; + +/* Profile summary for this object in current execuction. */ +static struct gcov_summary this_object; + +/* Merged profile summary for this program. */ +static struct gcov_summary program; + +/* Merged profile summary for this object. */ +static struct gcov_summary object; + +/* Record the position of summary info. */ +static gcov_position_t summary_pos = 0; + +/* Record the postion of eof. */ +static gcov_position_t eof_pos = 0; + +/* Number of chars in prefix to be stripped. */ +static int gcov_prefix_strip = 0; + +/* The length of path prefix. */ +static size_t prefix_length = 0; + +/* gi_filename is current object filename. + gi_filename_up points to the stripped filename. */ +static char *gi_filename, *gi_filename_up; + +static int gcov_open_by_filename (char * gi_filename); +static int gcov_exit_init (void); +static void gcov_dump_one_gcov (struct gcov_info *gi_ptr); + /* Make sure path component of the given FILENAME exists, create missing directories. FILENAME must be writable. Returns zero on success, or -1 if an error occurred. */ @@ -175,14 +252,47 @@ #endif } +/* Open a file with the specified name. */ + +static int +gcov_open_by_filename (char * gi_filename) +{ + if (!gcov_open (gi_filename)) + { + /* Open failed likely due to missed directory. + Create directory and retry to open file. */ + if (create_file_directory (gi_filename)) + { + gcov_error ("profiling:%s:Skip\n", gi_filename); + return -1; + } + if (!gcov_open (gi_filename)) + { + gcov_error ("profiling:%s:Cannot open\n", gi_filename); + return -1; + } + } + return 0; +} + + +/* Determine whether a counter is active. */ + +static inline int +gcov_counter_active (const struct gcov_info *info, unsigned int type) +{ + return (1 << type) & info->ctr_mask; +} + +#ifndef __GCOV_KERNEL__ /* Check if VERSION of the info block PTR matches libgcov one. Return 1 on success, or zero in case of versions mismatch. If FILENAME is not NULL, its value used for reporting purposes instead of value from the info block. */ static int -gcov_version (struct gcov_info *ptr, gcov_unsigned_t version, - const char *filename) +gcov_version (struct gcov_info *ptr __attribute__ ((unused)), + gcov_unsigned_t version, const char *filename) { if (version != GCOV_VERSION) { @@ -192,17 +302,17 @@ GCOV_UNSIGNED2STRING (e, GCOV_VERSION); if (filename) - fprintf (stderr, - "profiling:%s:Version mismatch - expected %.4s got %.4s\n", - filename, e, v); + gcov_error ("profiling:%s:Version mismatch - expected %.4s got %.4s\n", + filename, e, v); else - fprintf (stderr, - "profiling:Version mismatch - expected %.4s got %.4s\n", e, v); + gcov_error ("profiling:Version mismatch - expected %.4s got %.4s\n", e, v); return 0; } return 1; } +#define GCOV_GET_FILENAME gcov_strip_leading_dirs + /* Strip GCOV_PREFIX_STRIP levels of leading '/' from FILENAME and put the result into GI_FILENAME_UP. */ @@ -298,7 +408,7 @@ FILE *imports_file; size_t prefix_length, suffix_length; - gcov_suffix = getenv("GCOV_IMPORTS_SUFFIX"); + gcov_suffix = getenv ("GCOV_IMPORTS_SUFFIX"); if (!gcov_suffix || !strlen (gcov_suffix)) gcov_suffix = ".imports"; suffix_length = strlen (gcov_suffix); @@ -328,64 +438,26 @@ } } -/* Dump the coverage counts. We merge with existing counts when - possible, to avoid growing the .da files ad infinitum. We use this - program's checksum to make sure we only accumulate whole program - statistics to the correct summary. An object file might be embedded - in two separate programs, and we must keep the two program - summaries separate. */ +/* This function allocates the space to store current file name. */ static void -gcov_exit (void) +gcov_alloc_filename (void) { - struct gcov_info *gi_ptr; - struct gcov_summary this_program; - struct gcov_summary all; - struct gcov_ctr_summary *cs_ptr; - const struct gcov_ctr_info *ci_ptr; - unsigned t_ix; - gcov_unsigned_t c_num; - const char *gcov_prefix; - int gcov_prefix_strip = 0; - size_t prefix_length; - char *gi_filename, *gi_filename_up; - int dump_module_info = 0; + /* Get file name relocation prefix. Non-absolute values are ignored. */ + char *gcov_prefix = 0; - memset (&all, 0, sizeof (all)); - /* Find the totals for this execution. */ - memset (&this_program, 0, sizeof (this_program)); - for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) - { - ci_ptr = gi_ptr->counts; - for (t_ix = 0; t_ix < GCOV_COUNTERS_SUMMABLE; t_ix++) - { - if (!((1 << t_ix) & gi_ptr->ctr_mask)) - continue; + prefix_length = 0; + gcov_prefix_strip = 0; - cs_ptr = &this_program.ctrs[t_ix]; - cs_ptr->num += ci_ptr->num; - for (c_num = 0; c_num < ci_ptr->num; c_num++) - { - cs_ptr->sum_all += ci_ptr->values[c_num]; - if (cs_ptr->run_max < ci_ptr->values[c_num]) - cs_ptr->run_max = ci_ptr->values[c_num]; - } - ci_ptr++; - } - /* The IS_PRIMARY field is overloaded to indicate if this module - is FDO/LIPO. */ - dump_module_info |= gi_ptr->mod_info->is_primary; - } - { /* Check if the level of dirs to strip off specified. */ char *tmp = getenv ("GCOV_PREFIX_STRIP"); if (tmp) { - gcov_prefix_strip = atoi (tmp); - /* Do not consider negative values. */ - if (gcov_prefix_strip < 0) - gcov_prefix_strip = 0; + gcov_prefix_strip = atoi (tmp); + /* Do not consider negative values. */ + if (gcov_prefix_strip < 0) + gcov_prefix_strip = 0; } } /* Get file name relocation prefix. Non-absolute values are ignored. */ @@ -396,7 +468,7 @@ /* Remove an unnecessary trailing '/' */ if (IS_DIR_SEPARATOR (gcov_prefix[prefix_length - 1])) - prefix_length--; + prefix_length--; } else prefix_length = 0; @@ -408,259 +480,527 @@ gcov_prefix = "."; prefix_length = 1; } - /* Allocate and initialize the filename scratch space plus one. */ - gi_filename = (char *) alloca (prefix_length + gcov_max_filename + 2); + + /* Aelocate and initialize the filename scratch space. */ + gi_filename = (char *) malloc (prefix_length + gcov_max_filename + 2); if (prefix_length) memcpy (gi_filename, gcov_prefix, prefix_length); - gi_filename_up = gi_filename + prefix_length; +} - /* Now merge each file. */ +static void +gcov_dump_module_info (void) +{ + struct gcov_info *gi_ptr; + + __gcov_compute_module_groups (); + + /* Now write out module group info. */ for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) + { + int error; + + gcov_strip_leading_dirs (prefix_length, gcov_prefix_strip, + gi_ptr->filename, gi_filename_up); + error = gcov_open_by_filename (gi_filename); + if (error != 0) + continue; + + /* Overwrite the zero word at the of the file. */ + gcov_rewrite (); + gcov_seek (gi_ptr->eof_pos); + + gcov_write_module_infos (gi_ptr); + gcov_truncate (); + + if ((error = gcov_close ())) + gcov_error (error < 0 ? "profiling:%s:Overflow writing\n" : + "profiling:%s:Error writing\n", + gi_filename); + gcov_write_import_file (gi_filename, gi_ptr); + } + __gcov_finalize_dyn_callgraph (); +} + +/* Dump the coverage counts. We merge with existing counts when + possible, to avoid growing the .da files ad infinitum. We use this + program's checksum to make sure we only accumulate whole program + statistics to the correct summary. An object file might be embedded + in two separate programs, and we must keep the two program + summaries separate. */ + +static void +gcov_exit (void) +{ + struct gcov_info *gi_ptr; + int dump_module_info; + + dump_module_info = gcov_exit_init (); + + + for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) + gcov_dump_one_gcov (gi_ptr); + + if (dump_module_info) + gcov_dump_module_info (); + + free (gi_filename); +} + +/* Add a new object file onto the bb chain. Invoked automatically + when running an object file's global ctors. */ + +void +__gcov_init (struct gcov_info *info) +{ + if (!gcov_sampling_rate_initialized) { - struct gcov_summary this_object; - struct gcov_summary object, program; - gcov_type *values[GCOV_COUNTERS]; - const struct gcov_fn_info *fi_ptr; - unsigned fi_stride; - unsigned c_ix, f_ix, n_counts; - struct gcov_ctr_summary *cs_obj, *cs_tobj, *cs_prg, *cs_tprg, *cs_all; - int error = 0; - gcov_unsigned_t tag, length; - gcov_position_t summary_pos = 0; - gcov_position_t eof_pos = 0; - const char *fname, *s; + const char* env_value_str = getenv ("GCOV_SAMPLING_RATE"); + if (env_value_str) + { + int env_value_int = atoi(env_value_str); + if (env_value_int >= 1) + __gcov_sampling_rate = env_value_int; + } + gcov_sampling_rate_initialized = 1; + } - fname = gi_ptr->filename; + if (!info->version) + return; - memset (&this_object, 0, sizeof (this_object)); - memset (&object, 0, sizeof (object)); + if (gcov_version (info, info->version, 0)) + { + const char *ptr = info->filename; + gcov_unsigned_t crc32 = gcov_crc32; + size_t filename_length = strlen (info->filename); - gcov_strip_leading_dirs (prefix_length, gcov_prefix_strip, - gi_ptr->filename, gi_filename_up); + /* Refresh the longest file name information. */ + if (filename_length > gcov_max_filename) + gcov_max_filename = filename_length; - /* Totals for this object file. */ - ci_ptr = gi_ptr->counts; - for (t_ix = 0; t_ix < GCOV_COUNTERS_SUMMABLE; t_ix++) + /* Assign the module ID (starting at 1). */ + info->mod_info->ident = (++gcov_cur_module_id); + gcc_assert (EXTRACT_MODULE_ID_FROM_GLOBAL_ID (GEN_FUNC_GLOBAL_ID ( + info->mod_info->ident, 0)) + == info->mod_info->ident); + + do { - if (!((1 << t_ix) & gi_ptr->ctr_mask)) - continue; + unsigned ix; + gcov_unsigned_t value = *ptr << 24; - cs_ptr = &this_object.ctrs[t_ix]; - cs_ptr->num += ci_ptr->num; - for (c_num = 0; c_num < ci_ptr->num; c_num++) + for (ix = 8; ix--; value <<= 1) { - cs_ptr->sum_all += ci_ptr->values[c_num]; - if (cs_ptr->run_max < ci_ptr->values[c_num]) - cs_ptr->run_max = ci_ptr->values[c_num]; + gcov_unsigned_t feedback; + + feedback = (value ^ crc32) & 0x80000000 ? 0x04c11db7 : 0; + crc32 <<= 1; + crc32 ^= feedback; } - ci_ptr++; - } + } while (*ptr++); - c_ix = 0; - for (t_ix = 0; t_ix < GCOV_COUNTERS; t_ix++) - if ((1 << t_ix) & gi_ptr->ctr_mask) + gcov_crc32 = crc32; + + if (!__gcov_list) + atexit (gcov_exit); + + info->next = __gcov_list; + __gcov_list = info; + } + info->version = 0; +} + +/* Called before fork or exec - write out profile information gathered so + far and reset it to zero. This avoids duplication or loss of the + profile information gathered so far. */ + +void +__gcov_flush (void) +{ + const struct gcov_info *gi_ptr; + + gcov_exit (); + for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) + { + unsigned t_ix; + const struct gcov_ctr_info *ci_ptr; + + for (t_ix = 0, ci_ptr = gi_ptr->counts; t_ix != GCOV_COUNTERS; t_ix++) + if (gcov_counter_active (gi_ptr, t_ix)) { - values[c_ix] = gi_ptr->counts[c_ix].values; - if (t_ix == GCOV_COUNTER_ICALL_TOPNV) - gcov_sort_icall_topn_counter (&gi_ptr->counts[c_ix]); - c_ix++; + memset (ci_ptr->values, 0, sizeof (gcov_type) * ci_ptr->num); + ci_ptr++; } + } +} - /* Calculate the function_info stride. This depends on the - number of counter types being measured. */ - fi_stride = offsetof (struct gcov_fn_info, n_ctrs) - + c_ix * sizeof (unsigned); - if (__alignof__ (struct gcov_fn_info) > sizeof (unsigned)) - { - fi_stride += __alignof__ (struct gcov_fn_info) - 1; - fi_stride &= ~(__alignof__ (struct gcov_fn_info) - 1); - } +#else /* __GCOV_KERNEL__ */ - if (!gcov_open (gi_filename)) - { - /* Open failed likely due to missed directory. - Create directory and retry to open file. */ - if (create_file_directory (gi_filename)) - { - fprintf (stderr, "profiling:%s:Skip\n", gi_filename); - continue; - } - if (!gcov_open (gi_filename)) - { - fprintf (stderr, "profiling:%s:Cannot open\n", gi_filename); - continue; - } - } +#define GCOV_GET_FILENAME gcov_get_filename - tag = gcov_read_unsigned (); - if (tag) - { - /* Merge data from file. */ - if (tag != GCOV_DATA_MAGIC) - { - fprintf (stderr, "profiling:%s:Not a gcov data file\n", - gi_filename); - goto read_fatal; - } - length = gcov_read_unsigned (); - if (!gcov_version (gi_ptr, length, gi_filename)) - goto read_fatal; +/* Copy the filename to the buffer. */ - length = gcov_read_unsigned (); - if (length != gi_ptr->stamp) - /* Read from a different compilation. Overwrite the file. */ - goto rewrite; +static inline void +gcov_get_filename (int prefix_length __attribute__ ((unused)), + int gcov_prefix_strip __attribute__ ((unused)), + const char *filename, char *gi_filename_up) +{ + strcpy (gi_filename_up, filename); +} - /* Merge execution counts for each function. */ - for (f_ix = 0; f_ix < gi_ptr->n_functions; f_ix++) - { - fi_ptr = (const struct gcov_fn_info *) - ((const char *) gi_ptr->functions + f_ix * fi_stride); - tag = gcov_read_unsigned (); - length = gcov_read_unsigned (); +/* Sort the profile counters for all indirect call sites. Counters + for each call site are allocated in array COUNTERS. */ - /* Check function. */ - if (tag != GCOV_TAG_FUNCTION - || length != GCOV_TAG_FUNCTION_LENGTH - || gcov_read_unsigned () != fi_ptr->ident - || gcov_read_unsigned () != fi_ptr->lineno_checksum - || gcov_read_unsigned () != fi_ptr->cfg_checksum) - { - read_mismatch:; - fprintf (stderr, "profiling:%s:Merge mismatch for %s\n", - gi_filename, - f_ix + 1 ? "function" : "summaries"); - goto read_fatal; - } +static void +gcov_sort_icall_topn_counter (const struct gcov_ctr_info *counters) +{ + /* Empty */ +} - c_ix = 0; - for (t_ix = 0; t_ix < GCOV_COUNTERS; t_ix++) - { - gcov_merge_fn merge; +/* Reserves a buffer to store the name of the file being processed. */ +static char _kernel_gi_filename[520]; - if (!((1 << t_ix) & gi_ptr->ctr_mask)) - continue; +/* This function allocates the space to store current file name. */ - n_counts = fi_ptr->n_ctrs[c_ix]; - merge = gi_ptr->counts[c_ix].merge; +static void +gcov_alloc_filename (void) +{ + prefix_length = 0; + gcov_prefix_strip = 0; + gi_filename = _kernel_gi_filename; +} - tag = gcov_read_unsigned (); - length = gcov_read_unsigned (); - if (tag != GCOV_TAG_FOR_COUNTER (t_ix) - || length != GCOV_TAG_COUNTER_LENGTH (n_counts)) - goto read_mismatch; - (*merge) (values[c_ix], n_counts); - values[c_ix] += n_counts; - c_ix++; - } - if ((error = gcov_is_error ())) - goto read_error; - } +#endif /* __GCOV_KERNEL__ */ - f_ix = ~0u; - /* Check program & object summary */ - while (1) - { - int is_program; +/* Determine number of active counters in gcov_info INFO, + the counter arrays are stored in VALUES if the coming + value of VALUES !=0. If FLAG_SORT_ICALL_TOPN_COUNTER !=0, + the icall_topn_counter in INFO will be sorted. + Return: the number of active counter types. */ - eof_pos = gcov_position (); - tag = gcov_read_unsigned (); - if (!tag) - break; +static unsigned int +gcov_counter_array (const struct gcov_info *info, + gcov_type *values[GCOV_COUNTERS], + int flag_sort_icall_topn_counter) +{ + unsigned int i; + unsigned int result = 0; - length = gcov_read_unsigned (); - is_program = tag == GCOV_TAG_PROGRAM_SUMMARY; - if (length != GCOV_TAG_SUMMARY_LENGTH - || (!is_program && tag != GCOV_TAG_OBJECT_SUMMARY)) - goto read_mismatch; - gcov_read_summary (is_program ? &program : &object); - if ((error = gcov_is_error ())) - goto read_error; - if (is_program && program.checksum == gcov_crc32) - { - summary_pos = eof_pos; - goto rewrite; - } - } - } - goto rewrite; + for (i = 0; i < GCOV_COUNTERS; i++) { + if (gcov_counter_active (info, i)) + { + if (values) + values[result] = info->counts[result].values; + if (flag_sort_icall_topn_counter && + (i == GCOV_COUNTER_ICALL_TOPNV)) + gcov_sort_icall_topn_counter (&info->counts[result]); + result++; + } + } + return result; +} - read_error:; - fprintf (stderr, error < 0 ? "profiling:%s:Overflow merging\n" - : "profiling:%s:Error merging\n", gi_filename); +/* Compute object summary recored in gcov_info INFO. The result is + stored in OBJ_SUM. Note that the caller is responsible for + zeroing out OBJ_SUM, otherwise the summary is accumulated. */ - read_fatal:; - gcov_close (); - continue; +static void +gcov_object_summary (struct gcov_info *info, + struct gcov_summary *obj_sum) +{ + const struct gcov_ctr_info *ci_ptr; + struct gcov_ctr_summary *cs_ptr; + gcov_unsigned_t c_num; + unsigned t_ix; - rewrite:; - gcov_rewrite (); - if (!summary_pos) - memset (&program, 0, sizeof (program)); + /* Totals for this object file. */ + ci_ptr = info->counts; + for (t_ix = 0; t_ix < GCOV_COUNTERS_SUMMABLE; t_ix++) + { + if (!gcov_counter_active (info, t_ix)) + continue; - /* Merge the summaries. */ - f_ix = ~0u; - for (t_ix = 0; t_ix < GCOV_COUNTERS_SUMMABLE; t_ix++) - { - cs_obj = &object.ctrs[t_ix]; - cs_tobj = &this_object.ctrs[t_ix]; - cs_prg = &program.ctrs[t_ix]; - cs_tprg = &this_program.ctrs[t_ix]; - cs_all = &all.ctrs[t_ix]; + cs_ptr = &(obj_sum->ctrs[t_ix]); + cs_ptr->num += ci_ptr->num; + for (c_num = 0; c_num < ci_ptr->num; c_num++) + { + cs_ptr->sum_all += ci_ptr->values[c_num]; + if (cs_ptr->run_max < ci_ptr->values[c_num]) + cs_ptr->run_max = ci_ptr->values[c_num]; + } + ci_ptr++; + } +} - if ((1 << t_ix) & gi_ptr->ctr_mask) - { - if (!cs_obj->runs++) - cs_obj->num = cs_tobj->num; - else if (cs_obj->num != cs_tobj->num) - goto read_mismatch; - cs_obj->sum_all += cs_tobj->sum_all; - if (cs_obj->run_max < cs_tobj->run_max) - cs_obj->run_max = cs_tobj->run_max; - cs_obj->sum_max += cs_tobj->run_max; +/* Merge with existing gcda file in the same directory to avoid + excessive growthe of the files. */ - if (!cs_prg->runs++) - cs_prg->num = cs_tprg->num; - else if (cs_prg->num != cs_tprg->num) - goto read_mismatch; - cs_prg->sum_all += cs_tprg->sum_all; - if (cs_prg->run_max < cs_tprg->run_max) - cs_prg->run_max = cs_tprg->run_max; - cs_prg->sum_max += cs_tprg->run_max; - } - else if (cs_obj->num || cs_prg->num) - goto read_mismatch; +static int +gcov_merge_gcda_file (struct gcov_info *info, + gcov_type *values[GCOV_COUNTERS], + unsigned fi_stride) +{ + struct gcov_ctr_summary *cs_obj, *cs_tobj, *cs_prg, *cs_tprg, *cs_all; + unsigned t_ix, f_ix; - if (!cs_all->runs && cs_prg->runs) - memcpy (cs_all, cs_prg, sizeof (*cs_all)); - else if (!all.checksum - && (!GCOV_LOCKED || cs_all->runs == cs_prg->runs) - && memcmp (cs_all, cs_prg, sizeof (*cs_all))) - { - fprintf (stderr, "profiling:%s:Invocation mismatch - some data files may have been removed%s", - gi_filename, GCOV_LOCKED - ? "" : " or concurrent update without locking support"); - all.checksum = ~0u; - } - } +#ifndef __GCOV_KERNEL__ + const struct gcov_fn_info *fi_ptr; + unsigned c_ix, n_counts; + int error = 0; + gcov_unsigned_t tag, length; + tag = gcov_read_unsigned (); + if (tag) + { + /* Merge data from file. */ + if (tag != GCOV_DATA_MAGIC) + { + gcov_error ("profiling:%s:Not a gcov data file\n", gi_filename); + goto read_fatal; + } + length = gcov_read_unsigned (); + if (!gcov_version (info, length, gi_filename)) + goto read_fatal; + + length = gcov_read_unsigned (); + if (length != info->stamp) + /* Read from a different compilation. Overwrite the file. */ + goto rewrite; + + /* Merge execution counts for each function. */ + for (f_ix = 0; f_ix < info->n_functions; f_ix++) + { + fi_ptr = (const struct gcov_fn_info *) + ((const char *) info->functions + f_ix * fi_stride); + tag = gcov_read_unsigned (); + length = gcov_read_unsigned (); + + /* Check function. */ + if (tag != GCOV_TAG_FUNCTION + || length != GCOV_TAG_FUNCTION_LENGTH + || gcov_read_unsigned () != fi_ptr->ident + || gcov_read_unsigned () != fi_ptr->lineno_checksum + || gcov_read_unsigned () != fi_ptr->cfg_checksum) + goto read_mismatch; + + c_ix = 0; + for (t_ix = 0; t_ix < GCOV_COUNTERS; t_ix++) + { + gcov_merge_fn merge; + + if (!((1 << t_ix) & info->ctr_mask)) + continue; + + n_counts = fi_ptr->n_ctrs[c_ix]; + merge = info->counts[c_ix].merge; + + tag = gcov_read_unsigned (); + length = gcov_read_unsigned (); + if (tag != GCOV_TAG_FOR_COUNTER (t_ix) + || length != GCOV_TAG_COUNTER_LENGTH (n_counts)) + goto read_mismatch; + (*merge) (values[c_ix], n_counts); + values[c_ix] += n_counts; + c_ix++; + } + if ((error = gcov_is_error ())) + goto read_error; + } + + f_ix = ~0u; + /* Check program & object summary. */ + while (1) + { + int is_program; + + eof_pos = gcov_position (); + tag = gcov_read_unsigned (); + if (!tag) + break; + + length = gcov_read_unsigned (); + is_program = tag == GCOV_TAG_PROGRAM_SUMMARY; + if (length != GCOV_TAG_SUMMARY_LENGTH + || (!is_program && tag != GCOV_TAG_OBJECT_SUMMARY)) + goto read_mismatch; + gcov_read_summary (is_program ? &program : &object); + if ((error = gcov_is_error ())) + goto read_error; + if (is_program && program.checksum == gcov_crc32) + { + summary_pos = eof_pos; + goto rewrite; + } + } + } + + goto rewrite; + +read_error:; + gcov_error (error < 0 ? "profiling:%s:Overflow merging\n" + : "profiling:%s:Error merging\n", gi_filename); + goto read_fatal; + +#endif /* __GCOV_KERNEL__ */ + + goto rewrite; + +read_mismatch:; + gcov_error ("profiling:%s:Merge mismatch for %s\n", gi_filename, + f_ix + 1 ? "function" : "summaries"); + goto read_fatal; /* work-around the compiler warning */ + +read_fatal:; + gcov_close (); + return 1; + +rewrite:; + gcov_rewrite (); + if (!summary_pos) + memset (&program, 0, sizeof (program)); + + /* Merge the summaries. */ + f_ix = ~0u; + for (t_ix = 0; t_ix < GCOV_COUNTERS_SUMMABLE; t_ix++) + { + cs_obj = &object.ctrs[t_ix]; + cs_tobj = &this_object.ctrs[t_ix]; + cs_prg = &program.ctrs[t_ix]; + cs_tprg = &this_program.ctrs[t_ix]; + cs_all = &all.ctrs[t_ix]; + + if ((1 << t_ix) & info->ctr_mask) + { + if (!cs_obj->runs++) + cs_obj->num = cs_tobj->num; + else if (cs_obj->num != cs_tobj->num) + goto read_mismatch; + cs_obj->sum_all += cs_tobj->sum_all; + if (cs_obj->run_max < cs_tobj->run_max) + cs_obj->run_max = cs_tobj->run_max; + cs_obj->sum_max += cs_tobj->run_max; + + if (!cs_prg->runs++) + cs_prg->num = cs_tprg->num; + else if (cs_prg->num != cs_tprg->num) + goto read_mismatch; + cs_prg->sum_all += cs_tprg->sum_all; + if (cs_prg->run_max < cs_tprg->run_max) + cs_prg->run_max = cs_tprg->run_max; + cs_prg->sum_max += cs_tprg->run_max; + } + else if (cs_obj->num || cs_prg->num) + goto read_mismatch; + + if (!cs_all->runs && cs_prg->runs) + memcpy (cs_all, cs_prg, sizeof (*cs_all)); + else if (!all.checksum + && (!GCOV_LOCKED || cs_all->runs == cs_prg->runs) + && memcmp (cs_all, cs_prg, sizeof (*cs_all))) + { + gcov_error ("profiling:%s:Invocation mismatch - " + "some data files may have been removed%s", + gi_filename, GCOV_LOCKED + ? "" : " or concurrent update without locking support"); + all.checksum = ~0u; + } + } + + return 0; +} + +/* Calculate the function_info stride. This depends on the + number of counter types being measured. + NUM_COUNTER_TYPES is number of counter types recorded. + Return: the number of bytes for accessing next fn_info + (aligned to gcov_fn_info). */ + +static unsigned +gcov_compute_fi_stride (unsigned num_counter_types) +{ + unsigned fi_stride; + + fi_stride = offsetof (struct gcov_fn_info, n_ctrs) + + num_counter_types * sizeof (unsigned); + if (__alignof__ (struct gcov_fn_info) > sizeof (unsigned)) + { + fi_stride += __alignof__ (struct gcov_fn_info) - 1; + fi_stride &= ~(__alignof__ (struct gcov_fn_info) - 1); + } + return fi_stride; +} + +/* This function returns the size of gcda file to be written. Note + the size is in units of gcov_type. */ + +GCOV_LINKAGE unsigned +gcov_gcda_file_size (struct gcov_info *gi_ptr) +{ + unsigned size; + const struct gcov_fn_info *fi_ptr; + unsigned f_ix, t_ix, c_ix; + unsigned n_counts; + unsigned fi_stride; + gcov_type *values[GCOV_COUNTERS]; + + c_ix = gcov_counter_array (gi_ptr, values, 0); + fi_stride = gcov_compute_fi_stride (c_ix); + + /* GCOV_DATA_MAGIC, GCOV_VERSION and time_stamp. */ + size = 3; + + /* size for each function. */ + for (f_ix = 0; f_ix < gi_ptr->n_functions; f_ix++) + { + fi_ptr = (const struct gcov_fn_info *) + ((const char *) gi_ptr->functions + f_ix * fi_stride); + + size += 2 /* tag_length itself */ + + GCOV_TAG_FUNCTION_LENGTH; /* ident, lineno_cksum, cfg_cksm */ + c_ix = 0; for (t_ix = 0; t_ix < GCOV_COUNTERS; t_ix++) - if ((1 << t_ix) & gi_ptr->ctr_mask) - { - values[c_ix] = gi_ptr->counts[c_ix].values; - c_ix++; - } + { + if (!((1 << t_ix) & gi_ptr->ctr_mask)) + continue; - program.checksum = gcov_crc32; + n_counts = fi_ptr->n_ctrs[c_ix]; + size += 2 + GCOV_TAG_COUNTER_LENGTH (n_counts); + c_ix++; + } + } + /* Object summary. */ + size += 2 + GCOV_TAG_SUMMARY_LENGTH; + + /* Program summary. */ + size += 2 + GCOV_TAG_SUMMARY_LENGTH; + + size += 1; + + return size*4; +} + +/* Write profile data (including summary and module grouping information, + if available, to file. */ + +static void +gcov_write_gcda_file (struct gcov_info *gi_ptr, + unsigned fi_stride) +{ + const struct gcov_fn_info *fi_ptr; + gcov_type *values[GCOV_COUNTERS]; + unsigned t_ix, c_ix, f_ix, n_counts; + int error = 0; + /* Write out the data. */ gcov_write_tag_length (GCOV_DATA_MAGIC, GCOV_VERSION); gcov_write_unsigned (gi_ptr->stamp); + gcov_counter_array (gi_ptr, values, 0); + /* Write execution counts for each function. */ for (f_ix = 0; f_ix < gi_ptr->n_functions; f_ix++) - { + { fi_ptr = (const struct gcov_fn_info *) ((const char *) gi_ptr->functions + f_ix * fi_stride); @@ -689,12 +1029,13 @@ values[c_ix] = c_ptr; c_ix++; } - } + } /* Object file summary. */ gcov_write_summary (GCOV_TAG_OBJECT_SUMMARY, &object); /* Generate whole program statistics. */ + program.checksum = gcov_crc32; if (eof_pos) gcov_seek (eof_pos); gcov_write_summary (GCOV_TAG_PROGRAM_SUMMARY, &program); @@ -713,146 +1054,77 @@ gi_ptr->eof_pos = gcov_position (); if ((error = gcov_close ())) - fprintf (stderr, error < 0 ? + gcov_error (error < 0 ? "profiling:%s:Overflow writing\n" : "profiling:%s:Error writing\n", gi_filename); +} - } +/* Do some preparation work before calling the actual dumping + routine. + Return: 1 when module grouping info needs to be dumped, + 0 otherwise. */ - if (!dump_module_info) - return; +static int +gcov_exit_init (void) +{ + struct gcov_info *gi_ptr; + int dump_module_info = 0; - __gcov_compute_module_groups (); + dump_module_info = 0; + gcov_prefix_strip = 0; - /* Now write out module group info. */ - for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) + memset (&all, 0, sizeof (all)); + + /* Find the totals for this execution. */ + memset (&this_program, 0, sizeof (this_program)); + for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) { - int error; - gcov_strip_leading_dirs (prefix_length, gcov_prefix_strip, - gi_ptr->filename, gi_filename_up); + gcov_object_summary (gi_ptr, &this_program); - if (!gcov_open (gi_filename)) - { -#ifdef TARGET_POSIX_IO - /* Open failed likely due to missed directory. - Create directory and retry to open file. */ - if (create_file_directory (gi_filename)) - { - fprintf (stderr, "profiling:%s:Skip\n", gi_filename); - continue; - } -#endif - if (!gcov_open (gi_filename)) - { - fprintf (stderr, "profiling:%s:Cannot open\n", gi_filename); - continue; - } - } + /* The IS_PRIMARY field is overloaded to indicate if this module + is FDO/LIPO. */ + dump_module_info |= gi_ptr->mod_info->is_primary; + } - /* Overwrite the zero word at the of the file. */ - gcov_rewrite (); - gcov_seek (gi_ptr->eof_pos); + gcov_alloc_filename (); + gi_filename_up = gi_filename + prefix_length; - gcov_write_module_infos (gi_ptr); - gcov_truncate (); - - if ((error = gcov_close ())) - fprintf (stderr, error < 0 ? - "profiling:%s:Overflow writing\n" : - "profiling:%s:Error writing\n", - gi_filename); - gcov_write_import_file (gi_filename, gi_ptr); - } - __gcov_finalize_dyn_callgraph (); + return dump_module_info; } -/* Add a new object file onto the bb chain. Invoked automatically - when running an object file's global ctors. */ +/* Dump one entry in the gcov_info list (for one object). */ -void -__gcov_init (struct gcov_info *info) +static void +gcov_dump_one_gcov (struct gcov_info *gi_ptr) { - if (!gcov_sampling_rate_initialized) - { - const char* env_value_str = getenv ("GCOV_SAMPLING_RATE"); - if (env_value_str) - { - int env_value_int = atoi(env_value_str); - if (env_value_int >= 1) - __gcov_sampling_rate = env_value_int; - } - gcov_sampling_rate_initialized = 1; - } - if (!info->version) - return; - if (gcov_version (info, info->version, 0)) - { - const char *ptr = info->filename; - gcov_unsigned_t crc32 = gcov_crc32; - size_t filename_length = strlen(info->filename); + gcov_type *values[GCOV_COUNTERS]; + unsigned fi_stride; + unsigned c_ix; + int ret; - /* Refresh the longest file name information */ - if (filename_length > gcov_max_filename) - gcov_max_filename = filename_length; + memset (&this_object, 0, sizeof (this_object)); + memset (&object, 0, sizeof (object)); - /* Assign the module ID (starting at 1). */ - info->mod_info->ident = (++gcov_cur_module_id); - gcc_assert (EXTRACT_MODULE_ID_FROM_GLOBAL_ID (GEN_FUNC_GLOBAL_ID ( - info->mod_info->ident, 0)) - == info->mod_info->ident); + gcov_object_summary (gi_ptr, &this_object); - do - { - unsigned ix; - gcov_unsigned_t value = *ptr << 24; + c_ix = gcov_counter_array (gi_ptr, values, 1); - for (ix = 8; ix--; value <<= 1) - { - gcov_unsigned_t feedback; + fi_stride = gcov_compute_fi_stride (c_ix); - feedback = (value ^ crc32) & 0x80000000 ? 0x04c11db7 : 0; - crc32 <<= 1; - crc32 ^= feedback; - } - } - while (*ptr++); + GCOV_GET_FILENAME (prefix_length, gcov_prefix_strip, gi_ptr->filename, + gi_filename_up); - gcov_crc32 = crc32; + if (gcov_open_by_filename (gi_filename) == -1) + return; - if (!__gcov_list) - atexit (gcov_exit); + /* Now merge this file. */ + ret = gcov_merge_gcda_file (gi_ptr, values, fi_stride); + if (ret != 0 ) return; - info->next = __gcov_list; - __gcov_list = info; - } - info->version = 0; + gcov_write_gcda_file (gi_ptr, fi_stride); } -/* Called before fork or exec - write out profile information gathered so - far and reset it to zero. This avoids duplication or loss of the - profile information gathered so far. */ - -void -__gcov_flush (void) -{ - const struct gcov_info *gi_ptr; - - gcov_exit (); - for (gi_ptr = __gcov_list; gi_ptr; gi_ptr = gi_ptr->next) - { - unsigned t_ix; - const struct gcov_ctr_info *ci_ptr; - - for (t_ix = 0, ci_ptr = gi_ptr->counts; t_ix != GCOV_COUNTERS; t_ix++) - if ((1 << t_ix) & gi_ptr->ctr_mask) - { - memset (ci_ptr->values, 0, sizeof (gcov_type) * ci_ptr->num); - ci_ptr++; - } - } -} - #endif /* L_gcov */ #ifdef L_gcov_merge_add @@ -1556,4 +1828,99 @@ return execve (path, argv, envp); } #endif + +#ifdef __GCOV_KERNEL__ +/* + * Provide different implementation for the following functions: + * __gcov_init + * __gcov_exit + * + * Provide the following dummy merge functions: + * __gcov_merge_add + * __gcov_merge_single + * __gcov_merge_delta + * __gcov_merge_ior + * __gcov_merge_icall_topn + * __gcov_merge_dc + * __gcov_merge_reusedist + * + * Reuse the following functions: + * __gcov_interval_profiler() + * __gcov_pow2_profiler() + * __gcov_average_profiler() + * __gcov_ior_profiler() + * __gcov_one_value_profiler() + * __gcov_indirect_call_profiler() + * |-> __gcov_one_value_profiler_body() + * + * For LIPO: (TBD) + * Change slightly for the following functions: + * __gcov_merge_icall_topn + * __gcov_merge_dc + * + * Reuse the following functions: + * __gcov_direct_call_profiler() + * __gcov_indirect_call_topn_profiler() + * |-> __gcov_topn_value_profiler_body() + * + */ + +/* Current virual gcda file. This is for kernel use only. */ +gcov_kernel_vfile *gcov_current_file; + +/* Set current virutal gcda file. It needs to be set before dumping + profile data. */ + +void +gcov_set_vfile (gcov_kernel_vfile *file) +{ + gcov_current_file = file; +} + +/* Dump one entry in the gcov_info list (for one object) in kernel. */ + +void +gcov_kernel_dump_one_gcov (struct gcov_info *info) +{ + gcc_assert (gcov_current_file); + + gcov_exit_init (); + + gcov_dump_one_gcov (info); +} + +#define DUMMY_FUNC(func) \ +void func (gcov_type *counters __attribute__ ((unused)), \ + unsigned n_counters __attribute__ ((unused))) {} + +DUMMY_FUNC (__gcov_merge_add) +EXPORT_SYMBOL (__gcov_merge_add); + +DUMMY_FUNC (__gcov_merge_single) +EXPORT_SYMBOL (__gcov_merge_single); + +DUMMY_FUNC (__gcov_merge_delta) +EXPORT_SYMBOL (__gcov_merge_delta); + +DUMMY_FUNC(__gcov_merge_ior) +EXPORT_SYMBOL (__gcov_merge_ior); + +DUMMY_FUNC (__gcov_merge_icall_topn) +EXPORT_SYMBOL (__gcov_merge_icall_topn); + +DUMMY_FUNC (__gcov_merge_dc) +EXPORT_SYMBOL (__gcov_merge_dc); + +DUMMY_FUNC (__gcov_merge_reusedist) +EXPORT_SYMBOL (__gcov_merge_reusedist); + +EXPORT_SYMBOL (__gcov_average_profiler); +EXPORT_SYMBOL (__gcov_indirect_call_profiler); +EXPORT_SYMBOL (__gcov_interval_profiler); +EXPORT_SYMBOL (__gcov_ior_profiler); +EXPORT_SYMBOL (__gcov_one_value_profiler); +EXPORT_SYMBOL (__gcov_pow2_profiler); + +#endif /* __GCOV_KERNEL__ */ + #endif /* inhibit_libc */