mbox series

[v6,00/10] Improvements to memory usage by parse events

Message ID 20191107221428.168286-1-irogers@google.com
Headers show
Series Improvements to memory usage by parse events | expand

Message

Ian Rogers Nov. 7, 2019, 10:14 p.m. UTC
The parse events parser leaks memory for certain expressions as well
as allowing a char* to reference stack, heap or .rodata. This series
of patches improves the hygeine and adds free-ing operations to
reclaim memory in the parser in error and non-error situations.

The series of patches was generated with LLVM's address sanitizer and
libFuzzer:
https://llvm.org/docs/LibFuzzer.html
called on the parse_events function with randomly generated input. With
the patches no leaks or memory corruption issues were present.

The v6 patches address a C90 compilation issue.

The v5 patches add initial error print to the set, as requested by
Jiri Olsa. They also fix additional 2 missed frees in the patch
'before yyabort-ing free components' and remove a redundant new_str
variable from the patch 'add parse events handle error' as spotted by
Stephane Eranian.

The v4 patches address review comments from Jiri Olsa, turning a long
error message into a single warning, fixing the data type in a list
iterator and reordering patches.

The v3 patches address review comments from Jiri Olsa improving commit
messages, handling ENOMEM errors from strdup better, and removing a
printed warning if an invalid event is passed.

The v2 patches are preferable to an earlier proposed patch:
   perf tools: avoid reading out of scope array

Ian Rogers (10):
  perf tools: add parse events handle error
  perf tools: move ALLOC_LIST into a function
  perf tools: avoid a malloc for array events
  perf tools: splice events onto evlist even on error
  perf tools: ensure config and str in terms are unique
  perf tools: add destructors for parse event terms
  perf tools: before yyabort-ing free components
  perf tools: if pmu configuration fails free terms
  perf tools: add a deep delete for parse event terms
  perf tools: report initial event parsing error

 tools/perf/arch/powerpc/util/kvm-stat.c |   9 +-
 tools/perf/builtin-stat.c               |   2 +
 tools/perf/builtin-trace.c              |  16 +-
 tools/perf/tests/parse-events.c         |   3 +-
 tools/perf/util/metricgroup.c           |   2 +-
 tools/perf/util/parse-events.c          | 239 +++++++++++----
 tools/perf/util/parse-events.h          |   7 +
 tools/perf/util/parse-events.y          | 390 +++++++++++++++++-------
 tools/perf/util/pmu.c                   |  32 +-
 9 files changed, 511 insertions(+), 189 deletions(-)

Comments

Arnaldo Carvalho de Melo Nov. 7, 2019, 10:23 p.m. UTC | #1
Em Thu, Nov 07, 2019 at 02:14:18PM -0800, Ian Rogers escreveu:
> The parse events parser leaks memory for certain expressions as well
> as allowing a char* to reference stack, heap or .rodata. This series
> of patches improves the hygeine and adds free-ing operations to
> reclaim memory in the parser in error and non-error situations.
> 
> The series of patches was generated with LLVM's address sanitizer and
> libFuzzer:
> https://llvm.org/docs/LibFuzzer.html
> called on the parse_events function with randomly generated input. With
> the patches no leaks or memory corruption issues were present.
> 
> The v6 patches address a C90 compilation issue.

Please take a look at what is in my perf/core branch, to see what is
left, if something needs fixing, please send a patch on top of that,

Thanks,

- Arnaldo
 
> The v5 patches add initial error print to the set, as requested by
> Jiri Olsa. They also fix additional 2 missed frees in the patch
> 'before yyabort-ing free components' and remove a redundant new_str
> variable from the patch 'add parse events handle error' as spotted by
> Stephane Eranian.
> 
> The v4 patches address review comments from Jiri Olsa, turning a long
> error message into a single warning, fixing the data type in a list
> iterator and reordering patches.
> 
> The v3 patches address review comments from Jiri Olsa improving commit
> messages, handling ENOMEM errors from strdup better, and removing a
> printed warning if an invalid event is passed.
> 
> The v2 patches are preferable to an earlier proposed patch:
>    perf tools: avoid reading out of scope array
> 
> Ian Rogers (10):
>   perf tools: add parse events handle error
>   perf tools: move ALLOC_LIST into a function
>   perf tools: avoid a malloc for array events
>   perf tools: splice events onto evlist even on error
>   perf tools: ensure config and str in terms are unique
>   perf tools: add destructors for parse event terms
>   perf tools: before yyabort-ing free components
>   perf tools: if pmu configuration fails free terms
>   perf tools: add a deep delete for parse event terms
>   perf tools: report initial event parsing error
> 
>  tools/perf/arch/powerpc/util/kvm-stat.c |   9 +-
>  tools/perf/builtin-stat.c               |   2 +
>  tools/perf/builtin-trace.c              |  16 +-
>  tools/perf/tests/parse-events.c         |   3 +-
>  tools/perf/util/metricgroup.c           |   2 +-
>  tools/perf/util/parse-events.c          | 239 +++++++++++----
>  tools/perf/util/parse-events.h          |   7 +
>  tools/perf/util/parse-events.y          | 390 +++++++++++++++++-------
>  tools/perf/util/pmu.c                   |  32 +-
>  9 files changed, 511 insertions(+), 189 deletions(-)
> 
> -- 
> 2.24.0.432.g9d3f5f5b63-goog
Ian Rogers Nov. 8, 2019, 6:18 p.m. UTC | #2
On Thu, Nov 7, 2019 at 2:23 PM Arnaldo Carvalho de Melo
<arnaldo.melo@gmail.com> wrote:
>
> Em Thu, Nov 07, 2019 at 02:14:18PM -0800, Ian Rogers escreveu:
> > The parse events parser leaks memory for certain expressions as well
> > as allowing a char* to reference stack, heap or .rodata. This series
> > of patches improves the hygeine and adds free-ing operations to
> > reclaim memory in the parser in error and non-error situations.
> >
> > The series of patches was generated with LLVM's address sanitizer and
> > libFuzzer:
> > https://llvm.org/docs/LibFuzzer.html
> > called on the parse_events function with randomly generated input. With
> > the patches no leaks or memory corruption issues were present.
> >
> > The v6 patches address a C90 compilation issue.
>
> Please take a look at what is in my perf/core branch, to see what is
> left, if something needs fixing, please send a patch on top of that,

Thanks, just the last patch remaining. I resent it rebased on your
perf/core branch:
https://lkml.org/lkml/2019/11/8/1103
https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/log/?h=perf/core

Thanks,
Ian

> Thanks,
>
> - Arnaldo
>
> > The v5 patches add initial error print to the set, as requested by
> > Jiri Olsa. They also fix additional 2 missed frees in the patch
> > 'before yyabort-ing free components' and remove a redundant new_str
> > variable from the patch 'add parse events handle error' as spotted by
> > Stephane Eranian.
> >
> > The v4 patches address review comments from Jiri Olsa, turning a long
> > error message into a single warning, fixing the data type in a list
> > iterator and reordering patches.
> >
> > The v3 patches address review comments from Jiri Olsa improving commit
> > messages, handling ENOMEM errors from strdup better, and removing a
> > printed warning if an invalid event is passed.
> >
> > The v2 patches are preferable to an earlier proposed patch:
> >    perf tools: avoid reading out of scope array
> >
> > Ian Rogers (10):
> >   perf tools: add parse events handle error
> >   perf tools: move ALLOC_LIST into a function
> >   perf tools: avoid a malloc for array events
> >   perf tools: splice events onto evlist even on error
> >   perf tools: ensure config and str in terms are unique
> >   perf tools: add destructors for parse event terms
> >   perf tools: before yyabort-ing free components
> >   perf tools: if pmu configuration fails free terms
> >   perf tools: add a deep delete for parse event terms
> >   perf tools: report initial event parsing error
> >
> >  tools/perf/arch/powerpc/util/kvm-stat.c |   9 +-
> >  tools/perf/builtin-stat.c               |   2 +
> >  tools/perf/builtin-trace.c              |  16 +-
> >  tools/perf/tests/parse-events.c         |   3 +-
> >  tools/perf/util/metricgroup.c           |   2 +-
> >  tools/perf/util/parse-events.c          | 239 +++++++++++----
> >  tools/perf/util/parse-events.h          |   7 +
> >  tools/perf/util/parse-events.y          | 390 +++++++++++++++++-------
> >  tools/perf/util/pmu.c                   |  32 +-
> >  9 files changed, 511 insertions(+), 189 deletions(-)
> >
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
>
> --
>
> - Arnaldo
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20191107222315.GA7261%40kernel.org.