mbox series

[0/4] Store multiple values for single value profilers

Message ID cover.1559637554.git.mliska@suse.cz
Headers show
Series Store multiple values for single value profilers | expand

Message

Martin Liška June 4, 2019, 8:39 a.m. UTC
Hi.

It's becoming more common that a training run happens in parallel environment.
That can lead to a not reproducible builds caused by different order of merging
of .gcda files. So that I'm suggesting to store up to 4 values for HIST_TYPE_SINGLE_VALUE
and HIST_TYPE_INDIR_CALL on disk. If the capacity is exceeded the whole counter is
marked as unstable (not reproducible).

Patch can bootstrap on x86_64-linux-gnu and survives regression tests.

Ready to be installed?
Thanks,
Martin

marxin (4):
  Remove indirect call top N counter type.
  Implement N disk counters for single value and indirect call counters.
  Dump histograms only if present.
  Update a bit dump format.

 gcc/doc/invoke.texi       |   3 -
 gcc/gcov-counter.def      |   3 -
 gcc/gcov-io.h             |   9 +-
 gcc/ipa-profile.c         |  13 ++-
 gcc/params.def            |   8 --
 gcc/profile.c             |   1 -
 gcc/tree-profile.c        |  23 +---
 gcc/value-prof.c          | 224 ++++++++++++++++----------------------
 gcc/value-prof.h          |   4 +-
 libgcc/Makefile.in        |  10 +-
 libgcc/libgcov-driver.c   |  80 --------------
 libgcc/libgcov-merge.c    | 139 +++++++++--------------
 libgcc/libgcov-profiler.c | 176 ++----------------------------
 libgcc/libgcov-util.c     |  19 ----
 libgcc/libgcov.h          |  12 +-
 15 files changed, 179 insertions(+), 545 deletions(-)

Comments

Richard Biener June 5, 2019, 1:49 p.m. UTC | #1
On Tue, Jun 4, 2019 at 10:44 AM Martin Liska <mliska@suse.cz> wrote:
>
> Hi.
>
> It's becoming more common that a training run happens in parallel environment.
> That can lead to a not reproducible builds caused by different order of merging
> of .gcda files. So that I'm suggesting to store up to 4 values for HIST_TYPE_SINGLE_VALUE
> and HIST_TYPE_INDIR_CALL on disk. If the capacity is exceeded the whole counter is
> marked as unstable (not reproducible).
>
> Patch can bootstrap on x86_64-linux-gnu and survives regression tests.
>
> Ready to be installed?

Thanks for working on this, I hope Honza can review and approve it.  Does this
solve the issue of profiledbootstrap results being not reproducible?  (if you
fix genchecksum to not generate different checksums)

I suppose this would also apply to a GCC 9 tree?

Thanks,
Richard.

> Thanks,
> Martin
>
> marxin (4):
>   Remove indirect call top N counter type.
>   Implement N disk counters for single value and indirect call counters.
>   Dump histograms only if present.
>   Update a bit dump format.
>
>  gcc/doc/invoke.texi       |   3 -
>  gcc/gcov-counter.def      |   3 -
>  gcc/gcov-io.h             |   9 +-
>  gcc/ipa-profile.c         |  13 ++-
>  gcc/params.def            |   8 --
>  gcc/profile.c             |   1 -
>  gcc/tree-profile.c        |  23 +---
>  gcc/value-prof.c          | 224 ++++++++++++++++----------------------
>  gcc/value-prof.h          |   4 +-
>  libgcc/Makefile.in        |  10 +-
>  libgcc/libgcov-driver.c   |  80 --------------
>  libgcc/libgcov-merge.c    | 139 +++++++++--------------
>  libgcc/libgcov-profiler.c | 176 ++----------------------------
>  libgcc/libgcov-util.c     |  19 ----
>  libgcc/libgcov.h          |  12 +-
>  15 files changed, 179 insertions(+), 545 deletions(-)
>
> --
> 2.21.0
>
Martin Liška June 6, 2019, 8:23 a.m. UTC | #2
On 6/5/19 3:49 PM, Richard Biener wrote:
> On Tue, Jun 4, 2019 at 10:44 AM Martin Liska <mliska@suse.cz> wrote:
>>
>> Hi.
>>
>> It's becoming more common that a training run happens in parallel environment.
>> That can lead to a not reproducible builds caused by different order of merging
>> of .gcda files. So that I'm suggesting to store up to 4 values for HIST_TYPE_SINGLE_VALUE
>> and HIST_TYPE_INDIR_CALL on disk. If the capacity is exceeded the whole counter is
>> marked as unstable (not reproducible).
>>
>> Patch can bootstrap on x86_64-linux-gnu and survives regression tests.
>>
>> Ready to be installed?
> 
> Thanks for working on this, I hope Honza can review and approve it. 

Yes, he'll do it soon.

> Does this
> solve the issue of profiledbootstrap results being not reproducible?  (if you
> fix genchecksum to not generate different checksums)

Hopefully, but it needs to be tested.

> 
> I suppose this would also apply to a GCC 9 tree?

Yes, it applies smoothly. Would you like to see it backported to 9.2?

Martin

> 
> Thanks,
> Richard.
> 
>> Thanks,
>> Martin
>>
>> marxin (4):
>>   Remove indirect call top N counter type.
>>   Implement N disk counters for single value and indirect call counters.
>>   Dump histograms only if present.
>>   Update a bit dump format.
>>
>>  gcc/doc/invoke.texi       |   3 -
>>  gcc/gcov-counter.def      |   3 -
>>  gcc/gcov-io.h             |   9 +-
>>  gcc/ipa-profile.c         |  13 ++-
>>  gcc/params.def            |   8 --
>>  gcc/profile.c             |   1 -
>>  gcc/tree-profile.c        |  23 +---
>>  gcc/value-prof.c          | 224 ++++++++++++++++----------------------
>>  gcc/value-prof.h          |   4 +-
>>  libgcc/Makefile.in        |  10 +-
>>  libgcc/libgcov-driver.c   |  80 --------------
>>  libgcc/libgcov-merge.c    | 139 +++++++++--------------
>>  libgcc/libgcov-profiler.c | 176 ++----------------------------
>>  libgcc/libgcov-util.c     |  19 ----
>>  libgcc/libgcov.h          |  12 +-
>>  15 files changed, 179 insertions(+), 545 deletions(-)
>>
>> --
>> 2.21.0
>>
Richard Biener June 6, 2019, 9:30 a.m. UTC | #3
On Thu, Jun 6, 2019 at 10:23 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 6/5/19 3:49 PM, Richard Biener wrote:
> > On Tue, Jun 4, 2019 at 10:44 AM Martin Liska <mliska@suse.cz> wrote:
> >>
> >> Hi.
> >>
> >> It's becoming more common that a training run happens in parallel environment.
> >> That can lead to a not reproducible builds caused by different order of merging
> >> of .gcda files. So that I'm suggesting to store up to 4 values for HIST_TYPE_SINGLE_VALUE
> >> and HIST_TYPE_INDIR_CALL on disk. If the capacity is exceeded the whole counter is
> >> marked as unstable (not reproducible).
> >>
> >> Patch can bootstrap on x86_64-linux-gnu and survives regression tests.
> >>
> >> Ready to be installed?
> >
> > Thanks for working on this, I hope Honza can review and approve it.
>
> Yes, he'll do it soon.
>
> > Does this
> > solve the issue of profiledbootstrap results being not reproducible?  (if you
> > fix genchecksum to not generate different checksums)
>
> Hopefully, but it needs to be tested.
>
> >
> > I suppose this would also apply to a GCC 9 tree?
>
> Yes, it applies smoothly. Would you like to see it backported to 9.2?

No, but eventually into our package.

Richard.

> Martin
>
> >
> > Thanks,
> > Richard.
> >
> >> Thanks,
> >> Martin
> >>
> >> marxin (4):
> >>   Remove indirect call top N counter type.
> >>   Implement N disk counters for single value and indirect call counters.
> >>   Dump histograms only if present.
> >>   Update a bit dump format.
> >>
> >>  gcc/doc/invoke.texi       |   3 -
> >>  gcc/gcov-counter.def      |   3 -
> >>  gcc/gcov-io.h             |   9 +-
> >>  gcc/ipa-profile.c         |  13 ++-
> >>  gcc/params.def            |   8 --
> >>  gcc/profile.c             |   1 -
> >>  gcc/tree-profile.c        |  23 +---
> >>  gcc/value-prof.c          | 224 ++++++++++++++++----------------------
> >>  gcc/value-prof.h          |   4 +-
> >>  libgcc/Makefile.in        |  10 +-
> >>  libgcc/libgcov-driver.c   |  80 --------------
> >>  libgcc/libgcov-merge.c    | 139 +++++++++--------------
> >>  libgcc/libgcov-profiler.c | 176 ++----------------------------
> >>  libgcc/libgcov-util.c     |  19 ----
> >>  libgcc/libgcov.h          |  12 +-
> >>  15 files changed, 179 insertions(+), 545 deletions(-)
> >>
> >> --
> >> 2.21.0
> >>
>