Message ID | CY5PR21MB3542F5D5271CA9CEEE3C4EF9910A9@CY5PR21MB3542.namprd21.prod.outlook.com |
---|---|
State | New |
Headers | show |
Series | Fix autoprofiledbootstrap build | expand |
On 11/21/22 14:57, Eugene Rozenfeld via Gcc-patches wrote: > 1. Fix gcov version > 2. Don't attempt to create an autoprofile file for cc1 since cc1plus > (not cc1) is not invoked when building cc1 > 3. Fix documentation typo > > Tested on x86_64-pc-linux-gnu. > > gcc/ChangeLog: > > * c/Make-lang.in: Don't attempt to create an autoprofile file for cc1 > * cp/Make-lang.in: Fix gcov version > * lto/Make-lang.in: Fix gcov version > * doc/install.texi: Fix documentation typo Just to be 100% sure. While the compiler is built with cc1plus, various runtime libraries are still build with the C compiler and thus would use cc1. AFAICT it looks like we don't try to build the runtime libraries to get any data about the behavior of the C compiler. Can you confirm? Assuming that's correct, this is fine for the trunk. Thanks, Jeff
I took another look at this. We actually collect perf data when building the libraries. So, we have ./prev-gcc/perf.data, ./prev-libcpp/perf.data, ./prev-libiberty/perf.data, etc. But when creating gcov data for -fauto-profile build of cc1plus or cc1 we only use ./prev-gcc/perf.data . So, a better solution would be either having a single perf.data for all builds (gcc and libraries) or merging perf.data files before attempting autostagefeedback. What would you recommend? Thanks, Eugene -----Original Message----- From: Jeff Law <jeffreyalaw@gmail.com> Sent: Tuesday, November 22, 2022 12:01 PM To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; gcc-patches@gcc.gnu.org; Andi Kleen <ak@linux.intel.com> Subject: [EXTERNAL] Re: [PATCH] Fix autoprofiledbootstrap build [You don't often get email from jeffreyalaw@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] On 11/21/22 14:57, Eugene Rozenfeld via Gcc-patches wrote: > 1. Fix gcov version > 2. Don't attempt to create an autoprofile file for cc1 since cc1plus > (not cc1) is not invoked when building cc1 3. Fix documentation typo > > Tested on x86_64-pc-linux-gnu. > > gcc/ChangeLog: > > * c/Make-lang.in: Don't attempt to create an autoprofile file for cc1 > * cp/Make-lang.in: Fix gcov version > * lto/Make-lang.in: Fix gcov version > * doc/install.texi: Fix documentation typo Just to be 100% sure. While the compiler is built with cc1plus, various runtime libraries are still build with the C compiler and thus would use cc1. AFAICT it looks like we don't try to build the runtime libraries to get any data about the behavior of the C compiler. Can you confirm? Assuming that's correct, this is fine for the trunk. Thanks, Jeff
On 11/22/22 14:20, Eugene Rozenfeld wrote:
> I took another look at this. We actually collect perf data when building the libraries. So, we have ./prev-gcc/perf.data, ./prev-libcpp/perf.data, ./prev-libiberty/perf.data, etc. But when creating gcov data for -fauto-profile build of cc1plus or cc1 we only use ./prev-gcc/perf.data . So, a better solution would be either having a single perf.data for all builds (gcc and libraries) or merging perf.data files before attempting autostagefeedback. What would you recommend?
ISTM that if neither approach loses data, then they're functionally
equivalent -- meaning that we can select whichever is easier to wire
into our build system.
A single perf.data might serialize the build. So perhaps separate, then
merge right before autostagefeedback.
But I'm willing to go with whatever you think is best.
Jeff
Hi Jeff, I revived profile_merger tool in http://github.com/google/autofdo and re-worked the patch to merge profiles for compiling the libraries. Please take a look at the attached patch. Thanks, Eugene -----Original Message----- From: Jeff Law <jeffreyalaw@gmail.com> Sent: Tuesday, November 22, 2022 10:16 PM To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; gcc-patches@gcc.gnu.org; Andi Kleen <ak@linux.intel.com> Subject: Re: [EXTERNAL] Re: [PATCH] Fix autoprofiledbootstrap build [You don't often get email from jeffreyalaw@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] On 11/22/22 14:20, Eugene Rozenfeld wrote: > I took another look at this. We actually collect perf data when building the libraries. So, we have ./prev-gcc/perf.data, ./prev-libcpp/perf.data, ./prev-libiberty/perf.data, etc. But when creating gcov data for -fauto-profile build of cc1plus or cc1 we only use ./prev-gcc/perf.data . So, a better solution would be either having a single perf.data for all builds (gcc and libraries) or merging perf.data files before attempting autostagefeedback. What would you recommend? ISTM that if neither approach loses data, then they're functionally equivalent -- meaning that we can select whichever is easier to wire into our build system. A single perf.data might serialize the build. So perhaps separate, then merge right before autostagefeedback. But I'm willing to go with whatever you think is best. Jeff
Ping for https://gcc.gnu.org/pipermail/gcc-patches/2023-March/613974.html Thanks, Eugene -----Original Message----- From: Eugene Rozenfeld Sent: Tuesday, March 14, 2023 2:21 PM To: Jeff Law <jeffreyalaw@gmail.com>; gcc-patches@gcc.gnu.org; Andi Kleen <ak@linux.intel.com> Subject: RE: [EXTERNAL] Re: [PATCH] Fix autoprofiledbootstrap build Hi Jeff, I revived profile_merger tool in http://github.com/google/autofdo and re-worked the patch to merge profiles for compiling the libraries. Please take a look at the attached patch. Thanks, Eugene -----Original Message----- From: Jeff Law <jeffreyalaw@gmail.com> Sent: Tuesday, November 22, 2022 10:16 PM To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; gcc-patches@gcc.gnu.org; Andi Kleen <ak@linux.intel.com> Subject: Re: [EXTERNAL] Re: [PATCH] Fix autoprofiledbootstrap build [You don't often get email from jeffreyalaw@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] On 11/22/22 14:20, Eugene Rozenfeld wrote: > I took another look at this. We actually collect perf data when building the libraries. So, we have ./prev-gcc/perf.data, ./prev-libcpp/perf.data, ./prev-libiberty/perf.data, etc. But when creating gcov data for -fauto-profile build of cc1plus or cc1 we only use ./prev-gcc/perf.data . So, a better solution would be either having a single perf.data for all builds (gcc and libraries) or merging perf.data files before attempting autostagefeedback. What would you recommend? ISTM that if neither approach loses data, then they're functionally equivalent -- meaning that we can select whichever is easier to wire into our build system. A single perf.data might serialize the build. So perhaps separate, then merge right before autostagefeedback. But I'm willing to go with whatever you think is best. Jeff
On 3/14/23 15:21, Eugene Rozenfeld wrote: > From 1808fe371ab5618b7c0ce22c0dbecdaf593e516d Mon Sep 17 00:00:00 2001 > From: Eugene Rozenfeld<erozen@microsoft.com> > Date: Mon, 21 Nov 2022 13:33:38 -0800 > Subject: [PATCH] Fix autoprofiledbootstrap build > > 1. Fix gcov version > 2. Merge perf data collected when compiling the compiler and runtime libraries > 3. Fix documentation typo > > Tested on x86_64-pc-linux-gnu. > > gcc/ChangeLog: > > * Makefile.in: Define PROFILE_MERGER > * Makefile.tpl: Define PROFILE_MERGER > * c/Make-lang.in: Merge perf data collected when compiling cc1 and runtime libraries > * cp/Make-lang.in: Merge perf data collected when compiling cc1plus and runtime libraries > * lto/Make-lang.in: Merge perf data collected when compiling lto1 and runtime libraries > * doc/install.texi: Fix documentation typo OK jeff
diff --git a/gcc/c/Make-lang.in b/gcc/c/Make-lang.in index 9bd9c0ea123..ba33ec03bf0 100644 --- a/gcc/c/Make-lang.in +++ b/gcc/c/Make-lang.in @@ -62,12 +62,6 @@ c_OBJS = $(C_OBJS) cc1-checksum.o c/gccspec.o # Use strict warnings for this front end. c-warn = $(STRICT_WARN) -ifeq ($(if $(wildcard ../stage_current),$(shell cat \ - ../stage_current)),stageautofeedback) -$(C_OBJS): ALL_COMPILERFLAGS += -fauto-profile=cc1.fda -$(C_OBJS): cc1.fda -endif - # compute checksum over all object files and the options # re-use the checksum from the prev-final stage so it passes # the bootstrap comparison and allows comparing of the cc1 binary @@ -88,9 +82,6 @@ cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS) cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS) @$(call LINK_PROGRESS,$(INDEX.c),end) -cc1.fda: ../stage1-gcc/cc1$(exeext) ../prev-gcc/$(PERF_DATA) - $(CREATE_GCOV) -binary ../stage1-gcc/cc1$(exeext) -gcov cc1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1 - # # Build hooks: @@ -180,7 +171,6 @@ c.mostlyclean: -rm -f cc1$(exeext) -rm -f c/*$(objext) -rm -f c/*$(coverageexts) - -rm -f cc1.fda c.clean: c.distclean: -rm -f c/config.status c/Makefile @@ -201,7 +191,4 @@ c.stageprofile: stageprofile-start -mv c/*$(objext) stageprofile/c c.stagefeedback: stagefeedback-start -mv c/*$(objext) stagefeedback/c -c.autostageprofile: autostageprofile-start - -mv c/*$(objext) autostageprofile/c -c.autostagefeedback: autostagefeedback-start - -mv c/*$(objext) autostagefeedback/c + diff --git a/gcc/cp/Make-lang.in b/gcc/cp/Make-lang.in index 291835d326e..49e5cd66912 100644 --- a/gcc/cp/Make-lang.in +++ b/gcc/cp/Make-lang.in @@ -178,7 +178,7 @@ endif cp/name-lookup.o: $(srcdir)/cp/std-name-hint.h cc1plus.fda: ../stage1-gcc/cc1plus$(exeext) ../prev-gcc/$(PERF_DATA) - $(CREATE_GCOV) -binary ../stage1-gcc/cc1plus$(exeext) -gcov cc1plus.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1 + $(CREATE_GCOV) -binary ../stage1-gcc/cc1plus$(exeext) -gcov cc1plus.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 2 # # Build hooks: diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi index c1876f24a84..61a483bc410 100644 --- a/gcc/doc/install.texi +++ b/gcc/doc/install.texi @@ -3059,7 +3059,7 @@ It is recommended to only use GCC for this. On Linux/x86_64 hosts with some restrictions (no virtualization) it is also possible to do autofdo build with @samp{make -autoprofiledback}. This uses Linux perf to sample branches in the +autoprofiledbootstrap}. This uses Linux perf to sample branches in the binary and then rebuild it with feedback derived from the profile. Linux perf and the @code{autofdo} toolkit needs to be installed for this. diff --git a/gcc/lto/Make-lang.in b/gcc/lto/Make-lang.in index a2dcf0dfc12..3ee748489ac 100644 --- a/gcc/lto/Make-lang.in +++ b/gcc/lto/Make-lang.in @@ -106,7 +106,7 @@ $(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) $(lto2.prev) lto/lto-dump.o: $(LTO_OBJS) lto1.fda: ../prev-gcc/lto1$(exeext) ../prev-gcc/$(PERF_DATA) - $(CREATE_GCOV) -binary ../prev-gcc/lto1$(exeext) -gcov lto1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1 + $(CREATE_GCOV) -binary ../prev-gcc/lto1$(exeext) -gcov lto1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 2 # LTO testing is done as part of C/C++/Fortran etc. testing. check-lto: