From patchwork Thu Mar 7 13:34:57 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 225847 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 5D8422C03AD for ; Fri, 8 Mar 2013 00:35:21 +1100 (EST) Comment: DKIM? See http://www.dkim.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=gcc.gnu.org; s=default; x=1363268122; h=Comment: DomainKey-Signature:Received:Received:Received:Received:Date: From:To:Cc:Subject:In-Reply-To:Message-ID:References:User-Agent: MIME-Version:Content-Type:Mailing-List:Precedence:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:Sender: Delivered-To; bh=zga7Puhf26QAqvpkNoR53kTWb4k=; b=qMQdFG3KlWgV9vg OdB8Cjr68S6ww1gAyf9dv1xCvqKjiioO6RJ6/8WduQCydekFGucLiJ57Awc3W0Cj CDjuK8+ERKMo3k/Ke3MzbR5juosnMunszGtAjAx+btE+zEDvqtey/Ev0Y+wLh56U Zc5Dgrk2eIyNscmC53b6PeDZxW+I= Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=gcc.gnu.org; h=Received:Received:X-SWARE-Spam-Status:X-Spam-Check-By:Received:Received:Date:From:To:Cc:Subject:In-Reply-To:Message-ID:References:User-Agent:MIME-Version:Content-Type:Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:Sender:Delivered-To; b=a93+lFFrwISJKDmlV1UkR7051cQEK/ARCJFdqKtMQ2OGMxVbFgJL1vBn3+thAp CohmzbGZ/MapixriwTYUBrArmUFn9FLXkIqplCN1TuHSy5bNMdhVzKKkGtcRz3/m 3RXIeMqoWmKaBJXKarDVbT12SH+hrTFpUEMCEb3kCMB0I=; Received: (qmail 17907 invoked by alias); 7 Mar 2013 13:35:12 -0000 Received: (qmail 17782 invoked by uid 22791); 7 Mar 2013 13:35:11 -0000 X-SWARE-Spam-Status: No, hits=-6.5 required=5.0 tests=AWL, BAYES_00, KHOP_RCVD_UNTRUST, KHOP_THREADED, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from cantor2.suse.de (HELO mx2.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 07 Mar 2013 13:34:58 +0000 Received: from relay1.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 34D65A4E0C; Thu, 7 Mar 2013 14:34:57 +0100 (CET) Date: Thu, 7 Mar 2013 14:34:57 +0100 (CET) From: Richard Biener To: Marek Polacek Cc: gcc-patches@gcc.gnu.org Subject: Re: [PATCH] track heap usage with -fmem-report In-Reply-To: <20130307132022.GA18923@redhat.com> Message-ID: References: <20130307132022.GA18923@redhat.com> User-Agent: Alpine 2.00 (LNX 1167 2008-08-23) MIME-Version: 1.0 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org On Thu, 7 Mar 2013, Marek Polacek wrote: > On Thu, Mar 07, 2013 at 01:59:31PM +0100, Richard Biener wrote: > > > > This hacky patch tracks per-pass heap usage with -fmem-report > > using glibc malloc hooks (which are deprecated!? eh ... I can > > see no replacement?) > > The "replacement" is that we should intercept the malloc calls > instead (as those always use PLT). Sure, though that's not as easy to "disable" that instrumentation (I assume you mean using strong definitions in the GCC executable). Also you need to instrument more calls (well, calloc ...). A nicer replacement would be a way to intercept all mmap/munmap/sbrk syscalls (as my maxmem script does using strace ...). Possible with forking ourselves and stracing. Or maybe with a newer fancy kernel interface I don't know of. Anyway, here's a slightly updated version of the patch. TODO: also print GC peak memory Richard. Index: gcc/toplev.c =================================================================== --- gcc/toplev.c.orig 2013-03-07 14:00:15.739543469 +0100 +++ gcc/toplev.c 2013-03-07 14:16:56.609723870 +0100 @@ -74,6 +74,7 @@ along with GCC; see the file COPYING3. #include "gimple.h" #include "tree-ssa-alias.h" #include "plugin.h" +#include "tree-pass.h" #if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO) #include "dbxout.h" @@ -1783,6 +1784,7 @@ dump_memory_report (bool final) dump_ggc_loc_statistics (final); dump_alias_stats (stderr); dump_pta_stats (stderr); + dump_pass_memory_stats (stderr); } /* Clean up: close opened files, etc. */ @@ -1868,7 +1870,9 @@ do_compile (void) init_cgraph (); init_final (main_input_filename); coverage_init (aux_base_name); - statistics_init (); + statistics_init (); + if (mem_report) + instrument_malloc (); invoke_plugin_callbacks (PLUGIN_START_UNIT, NULL); timevar_stop (TV_PHASE_SETUP); Index: gcc/tree-pass.h =================================================================== --- gcc/tree-pass.h.orig 2013-03-07 14:00:15.739543469 +0100 +++ gcc/tree-pass.h 2013-03-07 14:16:03.312127130 +0100 @@ -78,6 +78,11 @@ struct opt_pass /* Flags indicating common sets things to do before and after. */ unsigned int todo_flags_start; unsigned int todo_flags_finish; + + size_t alloc_cnt; + size_t alloc_sum; + size_t alloc_curr; + size_t alloc_peak; }; /* Description of GIMPLE pass. */ @@ -557,4 +562,7 @@ extern void disable_pass (const char *); extern void enable_pass (const char *); extern void dump_passes (void); +extern void instrument_malloc (void); +extern void dump_pass_memory_stats (FILE *); + #endif /* GCC_TREE_PASS_H */ Index: gcc/passes.c =================================================================== --- gcc/passes.c.orig 2013-03-07 14:00:15.766543776 +0100 +++ gcc/passes.c 2013-03-07 14:15:48.515961538 +0100 @@ -24,6 +24,7 @@ along with GCC; see the file COPYING3. #include "config.h" #include "system.h" +#include #include "coretypes.h" #include "tm.h" #include "line-map.h" @@ -105,6 +106,158 @@ bool in_gimple_form; bool first_pass_instance; +struct malloc_chunk_head { + size_t x; + size_t size; +}; +typedef struct malloc_chunk_head* mchunkptr; +#define mem2chunk(mem) ((mchunkptr)((char*)(mem) - 2*sizeof(size_t))) + +static void (*gcc_old_free_hook) (void *ptr, const void *); +static void *(*gcc_old_malloc_hook) (size_t size, const void *); +static void *(*gcc_old_realloc_hook) (void *ptr, size_t size, const void *); + +static void *gcc_malloc_hook (size_t size, const void *caller); +static void *gcc_realloc_hook (void *ptr, size_t size, const void *caller); +static void gcc_free_hook (void *ptr, const void *caller); + +struct opt_pass mem_overall; +struct opt_pass mem_rest_of_comp; + +static void * +gcc_malloc_hook (size_t size, const void *caller) +{ + void *res; + __malloc_hook = gcc_old_malloc_hook; + __realloc_hook = gcc_old_realloc_hook; + __free_hook = gcc_old_free_hook; + if (gcc_old_malloc_hook != NULL) + res = gcc_old_malloc_hook (size, caller); + else + res = xmalloc (size); + __malloc_hook = gcc_malloc_hook; + __realloc_hook = gcc_realloc_hook; + __free_hook = gcc_free_hook; + struct opt_pass *p = current_pass ? current_pass : &mem_rest_of_comp; + /* The allocator allocates a chunk >= size. */ + size = res ? mem2chunk (res)->size : 0; + p->alloc_cnt++; + p->alloc_sum += size; + p->alloc_curr += size; + if ((ssize_t) p->alloc_curr > (ssize_t) p->alloc_peak) + p->alloc_peak = p->alloc_curr; + mem_overall.alloc_cnt++; + mem_overall.alloc_sum += size; + mem_overall.alloc_curr += size; + if ((ssize_t) mem_overall.alloc_curr > (ssize_t) mem_overall.alloc_peak) + mem_overall.alloc_peak = mem_overall.alloc_curr; + return res; +} + +static void * +gcc_realloc_hook (void *ptr, size_t size, const void *caller) +{ + size_t oldsize = ptr ? mem2chunk (ptr)->size : 0; + void *res; + __malloc_hook = gcc_old_malloc_hook; + __realloc_hook = gcc_old_realloc_hook; + __free_hook = gcc_old_free_hook; + if (gcc_old_realloc_hook != NULL) + res = gcc_old_realloc_hook (ptr, size, caller); + else + res = xrealloc (ptr, size); + __malloc_hook = gcc_malloc_hook; + __realloc_hook = gcc_realloc_hook; + __free_hook = gcc_free_hook; + struct opt_pass *p = current_pass ? current_pass : &mem_rest_of_comp; + /* The allocator allocates a chunk >= size. */ + size = res ? mem2chunk (res)->size : 0; + p->alloc_cnt++; + p->alloc_sum += (size - oldsize); + p->alloc_curr += (size - oldsize); + if ((ssize_t) p->alloc_curr > (ssize_t) p->alloc_peak) + p->alloc_peak = p->alloc_curr; + mem_overall.alloc_cnt++; + mem_overall.alloc_sum += (size - oldsize); + mem_overall.alloc_curr += (size - oldsize); + if ((ssize_t) mem_overall.alloc_curr > (ssize_t) mem_overall.alloc_peak) + mem_overall.alloc_peak = mem_overall.alloc_curr; + return res; +} + +static void +gcc_free_hook (void *ptr, const void *caller) +{ + struct opt_pass *p = current_pass ? current_pass : &mem_rest_of_comp; + size_t size = ptr ? mem2chunk (ptr)->size : 0; + p->alloc_curr -= size; + mem_overall.alloc_curr -= size; + __malloc_hook = gcc_old_malloc_hook; + __realloc_hook = gcc_old_realloc_hook; + __free_hook = gcc_old_free_hook; + if (gcc_old_free_hook != NULL) + gcc_old_free_hook (ptr, caller); + else + free (ptr); + __malloc_hook = gcc_malloc_hook; + __realloc_hook = gcc_realloc_hook; + __free_hook = gcc_free_hook; +} + +static void +dump_pass_memory_stats_1 (FILE *file, struct opt_pass *pass) +{ + do + { + if (pass->sub) + dump_pass_memory_stats_1 (file, pass->sub); + + if (pass->alloc_cnt != 0 + || pass->alloc_curr != 0) + { + fprintf (file, "%.14s:", pass->name); + if (strlen (pass->name) < 7) + fprintf (file, "\t"); + fprintf (file, "\t%zd\t%zd\t%zd\t%zd\n", + pass->alloc_cnt, + pass->alloc_sum / 1024, ((ssize_t) pass->alloc_curr) / 1024, + pass->alloc_peak / 1024); + } + pass = pass->next; + } + while (pass); +} + +void +dump_pass_memory_stats (FILE *file) +{ + fprintf (file, "\n\nPer pass non-gc memory allocation stats\n\n"); + fprintf (file, "Pass\t\t Cnt\t Total kB\t Leak kB\t Peak kB\n"); + mem_rest_of_comp.name = "(no pass)"; + dump_pass_memory_stats_1 (file, &mem_rest_of_comp); + dump_pass_memory_stats_1 (file, all_lowering_passes); + dump_pass_memory_stats_1 (file, all_small_ipa_passes); + dump_pass_memory_stats_1 (file, all_regular_ipa_passes); + dump_pass_memory_stats_1 (file, all_lto_gen_passes); + dump_pass_memory_stats_1 (file, all_late_ipa_passes); + dump_pass_memory_stats_1 (file, all_passes); + mem_overall.name = "TOTAL"; + dump_pass_memory_stats_1 (file, &mem_overall); + fprintf (file, "\n"); +} + +void +instrument_malloc (void) +{ + gcc_old_malloc_hook = __malloc_hook; + __malloc_hook = gcc_malloc_hook; + gcc_old_free_hook = __free_hook; + __free_hook = gcc_free_hook; + gcc_old_realloc_hook = __realloc_hook; + __realloc_hook = gcc_realloc_hook; +} + + /* This is called from various places for FUNCTION_DECL, VAR_DECL, and TYPE_DECL nodes. @@ -1965,6 +2118,9 @@ execute_function_todo (void *data) } else if (flags & TODO_verify_stmts) verify_gimple_in_cfg (cfun); + if (current_loops + && cfun->curr_properties & PROP_loops) + verify_loop_structure (); if (flags & TODO_verify_flow) verify_flow_info (); if (current_loops && loops_state_satisfies_p (LOOP_CLOSED_SSA))