diff mbox

[03/21] PR jit/63854: Fix memory leaks within context/pass_manager/dump_manager

Message ID 1416393981-39626-4-git-send-email-dmalcolm@redhat.com
State New
Headers show

Commit Message

David Malcolm Nov. 19, 2014, 10:46 a.m. UTC
We weren't cleaning up the context, pass_manager or dump_manager.

An earlier version of the context and pass_manager classes had them
allocated in the GC-heap, but they were moved to the system heap
before that patch was committed; they were never cleaned up, so e.g.
all passes leak on each in-process iteration.

Cleaning all of this up fixes the largest of the leaks in the PR,
about 60KB per iteration:

==57820== 80,560 (88 direct, 80,472 indirect) bytes in 1 blocks are definitely lost in loss record 913 of 917
==57820==    at 0x4A06965: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==57820==    by 0x536A462: make_pass_dce(gcc::context*) (tree-ssa-dce.c:1703)
==57820==    by 0x518B8DA: gcc::pass_manager::pass_manager(gcc::context*) (pass-instances.def:173)
==57820==    by 0x4E8D6D9: gcc::context::context() (context.c:37)
==57820==    by 0x4E2ED19: toplev::main(int, char**) (toplev.c:1211)
==57820==    by 0x4E43186: gcc::jit::playback::context::compile() (jit-playback.c:1615)
==57820==    by 0x4E4018D: gcc::jit::recording::context::compile() (jit-recording.c:861)
==57820==    by 0x401CA4: test_jit (harness.h:190)
==57820==    by 0x401D88: main (harness.h:232)
==57820==
==57820== 161,488 (352 direct, 161,136 indirect) bytes in 4 blocks are definitely lost in loss record 917 of 917
==57820==    at 0x4A06965: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==57820==    by 0x5553102: make_pass_insert_vzeroupper(gcc::context*) (i386.c:2581)
==57820==    by 0x55571DF: ix86_option_override() (i386.c:4325)
==57820==    by 0x4E2EEF4: toplev::main(int, char**) (toplev.c:1295)
==57820==    by 0x4E43186: gcc::jit::playback::context::compile() (jit-playback.c:1615)
==57820==    by 0x4E4018D: gcc::jit::recording::context::compile() (jit-recording.c:861)
==57820==    by 0x401CA4: test_jit (harness.h:190)
==57820==    by 0x401D88: main (harness.h:232)

All of this additional cleanup happens within toplev::finalize, and
hence we don't call it from cc1, cc1plus, etc, only from libgccjit.so.

Note that some calls to register_pass_info were using a static:
  static struct register_pass_info
on the stack e.g.:

 opt_pass *pass_handle_trap_shadows = make_pass_handle_trap_shadows (g);
 static struct register_pass_info handle_trap_shadows_info
   = { pass_handle_trap_shadows, "eh_ranges",
       1, PASS_POS_INSERT_AFTER
     };

This led to crashes on the 2nd iteration: the on-stack struct was only
being built on the first iteration, using the result of the first call
to the "make_pass_" function.  Subsequent calls would thus be
registering a freed pass, giving a use-after-free error (which was
previously hidden because we weren't freeing them).

The fix is to remove the "static" from such structs.

This also fixes a leak of strings within dump_file_info for passes.

gcc/ChangeLog:
	PR jit/63854
	* config/alpha/alpha.c (alpha_option_override): Remove static from
	"handle_trap_shadows_info" and "align_insns_info".
	* config/i386/i386.c (ix86_option_override): Likewise for
	"insert_vzeroupper_info".
	* config/rl78/rl78.c (rl78_asm_file_start): Likewise for
	"rl78_devirt_info" and "rl78_move_elim_info".
	* config/rs6000/rs6000.c (rs6000_option_override): Likewise for
	"analyze_swaps_info".
	* context.c (gcc::context::~context): New.
	* context.h (gcc::context::~context): New.
	* dumpfile.c (dump_files): Add "false" initializers for new field
	"owns_strings".
	(gcc::dump_manager::~dump_manager): New.
        (gcc::dump_manager::dump_register): Add param "take_ownership".
	* dumpfile.h (struct dump_file_info): Add field "owns_strings".
	(gcc::dump_manager::~dump_manager): New.
        (gcc::dump_manager::dump_register): Add param "take_ownership".
	* pass_manager.h (gcc::pass_manager::operator delete): New.
	(gcc::pass_manager::~pass_manager): New.
	* passes.c (pass_manager::register_one_dump_file): Pass "true" to
	new "owns_strings" argument to dump_register.
	(pass_manager::operator delete): New.
	(delete_pass_tree): New function.
	(pass_manager::~pass_manager): New.
	* statistics.c (statistics_early_init): Pass "false" to
	new "owns_strings" argument to dump_register.
	* toplev.c (toplev::finalize): Clean up the context and thus the
	things it owns.
---
 gcc/config/alpha/alpha.c   |  4 ++--
 gcc/config/i386/i386.c     |  2 +-
 gcc/config/rl78/rl78.c     |  4 ++--
 gcc/config/rs6000/rs6000.c |  2 +-
 gcc/context.c              |  6 ++++++
 gcc/context.h              |  1 +
 gcc/dumpfile.c             | 47 ++++++++++++++++++++++++++++++++++------------
 gcc/dumpfile.h             | 11 ++++++++++-
 gcc/pass_manager.h         |  2 ++
 gcc/passes.c               | 39 +++++++++++++++++++++++++++++++++++++-
 gcc/statistics.c           |  3 ++-
 gcc/toplev.c               |  4 ++++
 12 files changed, 104 insertions(+), 21 deletions(-)

Comments

Jeff Law Nov. 19, 2014, 6:22 p.m. UTC | #1
On 11/19/14 03:46, David Malcolm wrote:
> We weren't cleaning up the context, pass_manager or dump_manager.
>
> An earlier version of the context and pass_manager classes had them
> allocated in the GC-heap, but they were moved to the system heap
> before that patch was committed; they were never cleaned up, so e.g.
> all passes leak on each in-process iteration.
>
> Cleaning all of this up fixes the largest of the leaks in the PR,
> about 60KB per iteration:
>
> ==57820== 80,560 (88 direct, 80,472 indirect) bytes in 1 blocks are definitely lost in loss record 913 of 917
> ==57820==    at 0x4A06965: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==57820==    by 0x536A462: make_pass_dce(gcc::context*) (tree-ssa-dce.c:1703)
> ==57820==    by 0x518B8DA: gcc::pass_manager::pass_manager(gcc::context*) (pass-instances.def:173)
> ==57820==    by 0x4E8D6D9: gcc::context::context() (context.c:37)
> ==57820==    by 0x4E2ED19: toplev::main(int, char**) (toplev.c:1211)
> ==57820==    by 0x4E43186: gcc::jit::playback::context::compile() (jit-playback.c:1615)
> ==57820==    by 0x4E4018D: gcc::jit::recording::context::compile() (jit-recording.c:861)
> ==57820==    by 0x401CA4: test_jit (harness.h:190)
> ==57820==    by 0x401D88: main (harness.h:232)
> ==57820==
> ==57820== 161,488 (352 direct, 161,136 indirect) bytes in 4 blocks are definitely lost in loss record 917 of 917
> ==57820==    at 0x4A06965: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==57820==    by 0x5553102: make_pass_insert_vzeroupper(gcc::context*) (i386.c:2581)
> ==57820==    by 0x55571DF: ix86_option_override() (i386.c:4325)
> ==57820==    by 0x4E2EEF4: toplev::main(int, char**) (toplev.c:1295)
> ==57820==    by 0x4E43186: gcc::jit::playback::context::compile() (jit-playback.c:1615)
> ==57820==    by 0x4E4018D: gcc::jit::recording::context::compile() (jit-recording.c:861)
> ==57820==    by 0x401CA4: test_jit (harness.h:190)
> ==57820==    by 0x401D88: main (harness.h:232)
>
> All of this additional cleanup happens within toplev::finalize, and
> hence we don't call it from cc1, cc1plus, etc, only from libgccjit.so.
>
> Note that some calls to register_pass_info were using a static:
>    static struct register_pass_info
> on the stack e.g.:
>
>   opt_pass *pass_handle_trap_shadows = make_pass_handle_trap_shadows (g);
>   static struct register_pass_info handle_trap_shadows_info
>     = { pass_handle_trap_shadows, "eh_ranges",
>         1, PASS_POS_INSERT_AFTER
>       };
>
> This led to crashes on the 2nd iteration: the on-stack struct was only
> being built on the first iteration, using the result of the first call
> to the "make_pass_" function.  Subsequent calls would thus be
> registering a freed pass, giving a use-after-free error (which was
> previously hidden because we weren't freeing them).
>
> The fix is to remove the "static" from such structs.
>
> This also fixes a leak of strings within dump_file_info for passes.
>
> gcc/ChangeLog:
> 	PR jit/63854
> 	* config/alpha/alpha.c (alpha_option_override): Remove static from
> 	"handle_trap_shadows_info" and "align_insns_info".
> 	* config/i386/i386.c (ix86_option_override): Likewise for
> 	"insert_vzeroupper_info".
> 	* config/rl78/rl78.c (rl78_asm_file_start): Likewise for
> 	"rl78_devirt_info" and "rl78_move_elim_info".
> 	* config/rs6000/rs6000.c (rs6000_option_override): Likewise for
> 	"analyze_swaps_info".
> 	* context.c (gcc::context::~context): New.
> 	* context.h (gcc::context::~context): New.
> 	* dumpfile.c (dump_files): Add "false" initializers for new field
> 	"owns_strings".
> 	(gcc::dump_manager::~dump_manager): New.
>          (gcc::dump_manager::dump_register): Add param "take_ownership".
> 	* dumpfile.h (struct dump_file_info): Add field "owns_strings".
> 	(gcc::dump_manager::~dump_manager): New.
>          (gcc::dump_manager::dump_register): Add param "take_ownership".
> 	* pass_manager.h (gcc::pass_manager::operator delete): New.
> 	(gcc::pass_manager::~pass_manager): New.
> 	* passes.c (pass_manager::register_one_dump_file): Pass "true" to
> 	new "owns_strings" argument to dump_register.
> 	(pass_manager::operator delete): New.
> 	(delete_pass_tree): New function.
> 	(pass_manager::~pass_manager): New.
> 	* statistics.c (statistics_early_init): Pass "false" to
> 	new "owns_strings" argument to dump_register.
> 	* toplev.c (toplev::finalize): Clean up the context and thus the
> 	things it owns.
You could strdup the strings coming out of the tables, then remove the 
take_ownership stuff.    There's an obvious tradeoff there, memory for a 
bit more simplicity.

I'm OK with the patch as-is.


jeff
diff mbox

Patch

diff --git a/gcc/config/alpha/alpha.c b/gcc/config/alpha/alpha.c
index 05db3dc..1c89288 100644
--- a/gcc/config/alpha/alpha.c
+++ b/gcc/config/alpha/alpha.c
@@ -399,13 +399,13 @@  alpha_option_override (void)
   };
 
   opt_pass *pass_handle_trap_shadows = make_pass_handle_trap_shadows (g);
-  static struct register_pass_info handle_trap_shadows_info
+  struct register_pass_info handle_trap_shadows_info
     = { pass_handle_trap_shadows, "eh_ranges",
 	1, PASS_POS_INSERT_AFTER
       };
 
   opt_pass *pass_align_insns = make_pass_align_insns (g);
-  static struct register_pass_info align_insns_info
+  struct register_pass_info align_insns_info
     = { pass_align_insns, "shorten",
 	1, PASS_POS_INSERT_BEFORE
       };
diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c
index 3166e03..784982d 100644
--- a/gcc/config/i386/i386.c
+++ b/gcc/config/i386/i386.c
@@ -4323,7 +4323,7 @@  static void
 ix86_option_override (void)
 {
   opt_pass *pass_insert_vzeroupper = make_pass_insert_vzeroupper (g);
-  static struct register_pass_info insert_vzeroupper_info
+  struct register_pass_info insert_vzeroupper_info
     = { pass_insert_vzeroupper, "reload",
 	1, PASS_POS_INSERT_AFTER
       };
diff --git a/gcc/config/rl78/rl78.c b/gcc/config/rl78/rl78.c
index 7b85cf8..86d2992 100644
--- a/gcc/config/rl78/rl78.c
+++ b/gcc/config/rl78/rl78.c
@@ -284,7 +284,7 @@  rl78_asm_file_start (void)
     }
 
   opt_pass *rl78_devirt_pass = make_pass_rl78_devirt (g);
-  static struct register_pass_info rl78_devirt_info =
+  struct register_pass_info rl78_devirt_info =
     {
       rl78_devirt_pass,
       "pro_and_epilogue",
@@ -293,7 +293,7 @@  rl78_asm_file_start (void)
     };
 
   opt_pass *rl78_move_elim_pass = make_pass_rl78_move_elim (g);
-  static struct register_pass_info rl78_move_elim_info =
+  struct register_pass_info rl78_move_elim_info =
     {
       rl78_move_elim_pass,
       "bbro",
diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c
index 4f66840..506daa1 100644
--- a/gcc/config/rs6000/rs6000.c
+++ b/gcc/config/rs6000/rs6000.c
@@ -4202,7 +4202,7 @@  rs6000_option_override (void)
      It's convenient to do it here (like i386 does).  */
   opt_pass *pass_analyze_swaps = make_pass_analyze_swaps (g);
 
-  static struct register_pass_info analyze_swaps_info
+  struct register_pass_info analyze_swaps_info
     = { pass_analyze_swaps, "cse1", 1, PASS_POS_INSERT_BEFORE };
 
   register_pass (&analyze_swaps_info);
diff --git a/gcc/context.c b/gcc/context.c
index 9279be4..d132946 100644
--- a/gcc/context.c
+++ b/gcc/context.c
@@ -38,3 +38,9 @@  gcc::context::context ()
   m_dumps = new gcc::dump_manager ();
   m_passes = new gcc::pass_manager (this);
 }
+
+gcc::context::~context ()
+{
+  delete m_passes;
+  delete m_dumps;
+}
diff --git a/gcc/context.h b/gcc/context.h
index 689ae5a..2bf28a7 100644
--- a/gcc/context.h
+++ b/gcc/context.h
@@ -32,6 +32,7 @@  class context
 {
 public:
   context ();
+  ~context ();
 
   /* The flag shows if there are symbols to be streamed for offloading.  */
   bool have_offload;
diff --git a/gcc/dumpfile.c b/gcc/dumpfile.c
index fca7b51..c2cd89b 100644
--- a/gcc/dumpfile.c
+++ b/gcc/dumpfile.c
@@ -49,29 +49,29 @@  int dump_flags;
    TREE_DUMP_INDEX enumeration in dumpfile.h.  */
 static struct dump_file_info dump_files[TDI_end] =
 {
-  {NULL, NULL, NULL, NULL, NULL, NULL, NULL, 0, 0, 0, 0, 0, 0},
+  {NULL, NULL, NULL, NULL, NULL, NULL, NULL, 0, 0, 0, 0, 0, 0, false},
   {".cgraph", "ipa-cgraph", NULL, NULL, NULL, NULL, NULL, TDF_IPA,
-   0, 0, 0, 0, 0},
+   0, 0, 0, 0, 0, false},
   {".type-inheritance", "ipa-type-inheritance", NULL, NULL, NULL, NULL, NULL, TDF_IPA,
-   0, 0, 0, 0, 0},
+   0, 0, 0, 0, 0, false},
   {".tu", "translation-unit", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 1},
+   0, 0, 0, 0, 1, false},
   {".class", "class-hierarchy", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 2},
+   0, 0, 0, 0, 2, false},
   {".original", "tree-original", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 3},
+   0, 0, 0, 0, 3, false},
   {".gimple", "tree-gimple", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 4},
+   0, 0, 0, 0, 4, false},
   {".nested", "tree-nested", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 5},
+   0, 0, 0, 0, 5, false},
 #define FIRST_AUTO_NUMBERED_DUMP 6
 
   {NULL, "tree-all", NULL, NULL, NULL, NULL, NULL, TDF_TREE,
-   0, 0, 0, 0, 0},
+   0, 0, 0, 0, 0, false},
   {NULL, "rtl-all", NULL, NULL, NULL, NULL, NULL, TDF_RTL,
-   0, 0, 0, 0, 0},
+   0, 0, 0, 0, 0, false},
   {NULL, "ipa-all", NULL, NULL, NULL, NULL, NULL, TDF_IPA,
-   0, 0, 0, 0, 0},
+   0, 0, 0, 0, 0, false},
 };
 
 /* Define a name->number mapping for a dump flag value.  */
@@ -148,10 +148,32 @@  gcc::dump_manager::dump_manager ():
 {
 }
 
+gcc::dump_manager::~dump_manager ()
+{
+  for (size_t i = 0; i < m_extra_dump_files_in_use; i++)
+    {
+      dump_file_info *dfi = &m_extra_dump_files[i];
+      /* suffix, swtch, glob are statically allocated for the entries
+	 in dump_files, and for statistics, but are dynamically allocated
+	 for those for passes.  */
+      if (dfi->owns_strings)
+	{
+	  XDELETEVEC (const_cast <char *> (dfi->suffix));
+	  XDELETEVEC (const_cast <char *> (dfi->swtch));
+	  XDELETEVEC (const_cast <char *> (dfi->glob));
+	}
+      /* These, if non-NULL, are always dynamically allocated.  */
+      XDELETEVEC (const_cast <char *> (dfi->pfilename));
+      XDELETEVEC (const_cast <char *> (dfi->alt_filename));
+    }
+  XDELETEVEC (m_extra_dump_files);
+}
+
 unsigned int
 gcc::dump_manager::
 dump_register (const char *suffix, const char *swtch, const char *glob,
-	       int flags, int optgroup_flags)
+	       int flags, int optgroup_flags,
+	       bool take_ownership)
 {
   int num = m_next_dump++;
 
@@ -175,6 +197,7 @@  dump_register (const char *suffix, const char *swtch, const char *glob,
   m_extra_dump_files[count].pflags = flags;
   m_extra_dump_files[count].optgroup_flags = optgroup_flags;
   m_extra_dump_files[count].num = num;
+  m_extra_dump_files[count].owns_strings = take_ownership;
 
   return count + TDI_end;
 }
diff --git a/gcc/dumpfile.h b/gcc/dumpfile.h
index 75949b7..d650174 100644
--- a/gcc/dumpfile.h
+++ b/gcc/dumpfile.h
@@ -118,6 +118,9 @@  struct dump_file_info
   int pstate;                   /* state of pass-specific stream */
   int alt_state;                /* state of the -fopt-info stream */
   int num;                      /* dump file number */
+  bool owns_strings;            /* fields "suffix", "swtch", "glob" can be
+				   const strings, or can be dynamically
+				   allocated, needing free.  */
 };
 
 /* In dumpfile.c */
@@ -164,10 +167,16 @@  class dump_manager
 public:
 
   dump_manager ();
+  ~dump_manager ();
 
+  /* Register a dumpfile.
+
+     TAKE_OWNERSHIP determines whether callee takes ownership of strings
+     SUFFIX, SWTCH, and GLOB. */
   unsigned int
   dump_register (const char *suffix, const char *swtch, const char *glob,
-		 int flags, int optgroup_flags);
+		 int flags, int optgroup_flags,
+		 bool take_ownership);
 
   /* Return the dump_file_info for the given phase.  */
   struct dump_file_info *
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index 9f4d67b..82857a9 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -47,8 +47,10 @@  class pass_manager
 {
 public:
   void *operator new (size_t sz);
+  void operator delete (void *ptr);
 
   pass_manager (context *ctxt);
+  ~pass_manager ();
 
   void register_pass (struct register_pass_info *pass_info);
   void register_one_dump_file (opt_pass *pass);
diff --git a/gcc/passes.c b/gcc/passes.c
index c818d8a..f7a0170 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -776,7 +776,8 @@  pass_manager::register_one_dump_file (opt_pass *pass)
   if (optgroup_flags == OPTGROUP_NONE)
     optgroup_flags = OPTGROUP_OTHER;
   id = dumps->dump_register (dot_name, flag_name, glob_name, flags,
-			     optgroup_flags);
+			     optgroup_flags,
+			     true);
   set_pass_for_id (id, pass);
   full_name = concat (prefix, pass->name, num, NULL);
   register_pass_name (pass, full_name);
@@ -1527,6 +1528,12 @@  pass_manager::operator new (size_t sz)
   return xcalloc (1, sz);
 }
 
+void
+pass_manager::operator delete (void *ptr)
+{
+  free (ptr);
+}
+
 pass_manager::pass_manager (context *ctxt)
 : all_passes (NULL), all_small_ipa_passes (NULL), all_lowering_passes (NULL),
   all_regular_ipa_passes (NULL),
@@ -1584,6 +1591,36 @@  pass_manager::pass_manager (context *ctxt)
   register_dump_files (all_passes);
 }
 
+static void
+delete_pass_tree (opt_pass *pass)
+{
+  while (pass)
+    {
+      /* Recurse into child passes.  */
+      delete_pass_tree (pass->sub);
+
+      opt_pass *next = pass->next;
+
+      /* Delete this pass.  */
+      delete pass;
+
+      /* Iterate onto sibling passes.  */
+      pass = next;
+    }
+}
+
+pass_manager::~pass_manager ()
+{
+  XDELETEVEC (passes_by_id);
+
+  /* Call delete_pass_tree on each of the pass_lists.  */
+#define DEF_PASS_LIST(LIST) \
+    delete_pass_tree (*pass_lists[PASS_LIST_NO_##LIST]);
+  GCC_PASS_LISTS
+#undef DEF_PASS_LIST
+
+}
+
 /* If we are in IPA mode (i.e., current_function_decl is NULL), call
    function CALLBACK for every function in the call graph.  Otherwise,
    call CALLBACK on the current function.  */
diff --git a/gcc/statistics.c b/gcc/statistics.c
index b3e63de..0ceb2e9 100644
--- a/gcc/statistics.c
+++ b/gcc/statistics.c
@@ -270,7 +270,8 @@  statistics_early_init (void)
   gcc::dump_manager *dumps = g->get_dumps ();
   statistics_dump_nr = dumps->dump_register (".statistics", "statistics",
 					     "statistics", TDF_TREE,
-					     OPTGROUP_NONE);
+					     OPTGROUP_NONE,
+					     false);
 }
 
 /* Init the statistics.  */
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 4b4e568..876279f 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -2172,4 +2172,8 @@  toplev::finalize (void)
 
   finalize_options_struct (&global_options);
   finalize_options_struct (&global_options_set);
+
+  /* Clean up the context (and pass_manager etc). */
+  delete g;
+  g = NULL;
 }