Patchwork [2/2] Support slim LTO bootstrap

login
register
mail settings
Submitter Andi Kleen
Date Aug. 31, 2014, 2:51 p.m.
Message ID <1409496686-14681-3-git-send-email-andi@firstfloor.org>
Download mbox | patch
Permalink /patch/384542/
State New
Headers show

Comments

Andi Kleen - Aug. 31, 2014, 2:51 p.m.
From: Andi Kleen <ak@linux.intel.com>

Change the bootstrap-lto config file to use slim (non fat) LTO..
Speeds up the LTO bootstrap by ~18% on a 4 core system.

This requires using gcc-ar/ranlib in post stage 1 builds, so these
are passed to all sub builds.

v2: Change existing config file as requested by Honza.

config/:

2014-08-31  Andi Kleen  <ak@linux.intel.com>

	* bootstrap-lto.mk: Implement slim bootstrap.

/:

2014-08-31  Andi Kleen  <ak@linux.intel.com>

	* Makefile.tpl (POSTSTAGE1_HOST_EXPORTS): Add LTO_EXPORTS.
	POSTSTAGE1_FLAGS_TO_PASS):  Add LTO_FLAGS_TO_PASS.
	* Makefile.in: Regenerate.
---
 Makefile.in                  |  2 ++
 Makefile.tpl                 |  2 ++
 config/bootstrap-lto-slim.mk | 13 +++++++++++++
 config/bootstrap-lto.mk      | 16 +++++++++++-----
 4 files changed, 28 insertions(+), 5 deletions(-)
 create mode 100644 config/bootstrap-lto-slim.mk
Andi Kleen - Aug. 31, 2014, 3:25 p.m.
Andi Kleen <andi@firstfloor.org> writes:
> diff --git a/config/bootstrap-lto-slim.mk b/config/bootstrap-lto-slim.mk
> new file mode 100644
> index 0000000..9e065e1
> --- /dev/null
> +++ b/config/bootstrap-lto-slim.mk

This file was not supposed to be included anymore. I removed it in my
version. Instead the existing bootstrap-lto.mk has the same changes.

-andi
Richard Guenther - Sept. 1, 2014, 11:43 a.m.
On Sun, Aug 31, 2014 at 4:51 PM, Andi Kleen <andi@firstfloor.org> wrote:
> From: Andi Kleen <ak@linux.intel.com>
>
> Change the bootstrap-lto config file to use slim (non fat) LTO..
> Speeds up the LTO bootstrap by ~18% on a 4 core system.
>
> This requires using gcc-ar/ranlib in post stage 1 builds, so these
> are passed to all sub builds.
>
> v2: Change existing config file as requested by Honza.
>
> config/:
>
> 2014-08-31  Andi Kleen  <ak@linux.intel.com>
>
>         * bootstrap-lto.mk: Implement slim bootstrap.
> /:
>
> 2014-08-31  Andi Kleen  <ak@linux.intel.com>
>
>         * Makefile.tpl (POSTSTAGE1_HOST_EXPORTS): Add LTO_EXPORTS.
>         POSTSTAGE1_FLAGS_TO_PASS):  Add LTO_FLAGS_TO_PASS.
>         * Makefile.in: Regenerate.
> ---
>  Makefile.in                  |  2 ++
>  Makefile.tpl                 |  2 ++
>  config/bootstrap-lto-slim.mk | 13 +++++++++++++
>  config/bootstrap-lto.mk      | 16 +++++++++++-----
>  4 files changed, 28 insertions(+), 5 deletions(-)
>  create mode 100644 config/bootstrap-lto-slim.mk
>
> diff --git a/Makefile.in b/Makefile.in
> index add8cf6..d6105b3 100644
> --- a/Makefile.in
> +++ b/Makefile.in
> @@ -257,6 +257,7 @@ POSTSTAGE1_HOST_EXPORTS = \
>           $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \
>         CC_FOR_BUILD="$$CC"; export CC_FOR_BUILD; \
>         $(POSTSTAGE1_CXX_EXPORT) \
> +       $(LTO_EXPORTS) \
>         GNATBIND="$$r/$(HOST_SUBDIR)/prev-gcc/gnatbind"; export GNATBIND; \
>         LDFLAGS="$(POSTSTAGE1_LDFLAGS) $(BOOT_LDFLAGS)"; export LDFLAGS; \
>         HOST_LIBS="$(POSTSTAGE1_LIBS)"; export HOST_LIBS;
> @@ -828,6 +829,7 @@ POSTSTAGE1_FLAGS_TO_PASS = \
>         GNATBIND="$${GNATBIND}" \
>         LDFLAGS="$${LDFLAGS}" \
>         HOST_LIBS="$${HOST_LIBS}" \
> +       $(LTO_FLAGS_TO_PASS) \
>         "`echo 'ADAFLAGS=$(BOOT_ADAFLAGS)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
>
>  # Flags to pass down to makes which are built with the target environment.
> diff --git a/Makefile.tpl b/Makefile.tpl
> index 00dba36..f7c7e38 100644
> --- a/Makefile.tpl
> +++ b/Makefile.tpl
> @@ -260,6 +260,7 @@ POSTSTAGE1_HOST_EXPORTS = \
>           $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \
>         CC_FOR_BUILD="$$CC"; export CC_FOR_BUILD; \
>         $(POSTSTAGE1_CXX_EXPORT) \
> +       $(LTO_EXPORTS) \
>         GNATBIND="$$r/$(HOST_SUBDIR)/prev-gcc/gnatbind"; export GNATBIND; \
>         LDFLAGS="$(POSTSTAGE1_LDFLAGS) $(BOOT_LDFLAGS)"; export LDFLAGS; \
>         HOST_LIBS="$(POSTSTAGE1_LIBS)"; export HOST_LIBS;
> @@ -633,6 +634,7 @@ POSTSTAGE1_FLAGS_TO_PASS = \
>         GNATBIND="$${GNATBIND}" \
>         LDFLAGS="$${LDFLAGS}" \
>         HOST_LIBS="$${HOST_LIBS}" \
> +       $(LTO_FLAGS_TO_PASS) \
>         "`echo 'ADAFLAGS=$(BOOT_ADAFLAGS)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
>
>  # Flags to pass down to makes which are built with the target environment.
> diff --git a/config/bootstrap-lto-slim.mk b/config/bootstrap-lto-slim.mk
> new file mode 100644
> index 0000000..9e065e1
> --- /dev/null
> +++ b/config/bootstrap-lto-slim.mk
> @@ -0,0 +1,13 @@
> +# This option enables LTO for stage2 and stage3 in slim mode
> +
> +STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1
> +STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1
> +STAGEprofile_CFLAGS += -fno-lto
> +
> +# assumes the host supports the linker plugin
> +LTO_AR = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ar$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
> +LTO_RANLIB = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ranlib$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
> +

-ffat-lto-objects is automatically active for hosts not supporting the
linker plugin.  Is gcc-ar$(exeext) unconditionally built and found
on such hosts?  Will gcc-ar work without finding a linker plugin
on such hosts?

As I think the non-linker-plugin path is not really interesting the
patch is ok as-is.  Just be prepared to followup with a fix for
such hosts (darwin I suppose, and other non-ELF hosts).

Thanks,
Richard.

> +LTO_EXPORTS = AR="$(LTO_AR)"; export AR; \
> +             RANLIB="$(LTO_RANLIB)"; export RANLIB;
> +LTO_FLAGS_TO_PASS = AR="$(LTO_AR)" RANLIB="$(LTO_RANLIB)"
> diff --git a/config/bootstrap-lto.mk b/config/bootstrap-lto.mk
> index 27bad15..9e065e1 100644
> --- a/config/bootstrap-lto.mk
> +++ b/config/bootstrap-lto.mk
> @@ -1,7 +1,13 @@
> -# This option enables LTO for stage2 and stage3.
> -# FIXME: Our build system is not yet able to use gcc-ar wrapper, so we need
> -# to go with -ffat-lto-objects.
> +# This option enables LTO for stage2 and stage3 in slim mode
>
> -STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1 -ffat-lto-objects
> -STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1 -ffat-lto-objects
> +STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1
> +STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1
>  STAGEprofile_CFLAGS += -fno-lto
> +
> +# assumes the host supports the linker plugin
> +LTO_AR = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ar$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
> +LTO_RANLIB = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ranlib$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
> +
> +LTO_EXPORTS = AR="$(LTO_AR)"; export AR; \
> +             RANLIB="$(LTO_RANLIB)"; export RANLIB;
> +LTO_FLAGS_TO_PASS = AR="$(LTO_AR)" RANLIB="$(LTO_RANLIB)"
> --
> 2.0.4
>
Andi Kleen - Sept. 1, 2014, 3:04 p.m.
> -ffat-lto-objects is automatically active for hosts not supporting the
> linker plugin.  Is gcc-ar$(exeext) unconditionally built and found
> on such hosts?  Will gcc-ar work without finding a linker plugin
> on such hosts?

Currently it errors out. I suppose that could be turned into a warning

I think it's always built.

-Andi

Patch

diff --git a/Makefile.in b/Makefile.in
index add8cf6..d6105b3 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -257,6 +257,7 @@  POSTSTAGE1_HOST_EXPORTS = \
 	  $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \
 	CC_FOR_BUILD="$$CC"; export CC_FOR_BUILD; \
 	$(POSTSTAGE1_CXX_EXPORT) \
+	$(LTO_EXPORTS) \
 	GNATBIND="$$r/$(HOST_SUBDIR)/prev-gcc/gnatbind"; export GNATBIND; \
 	LDFLAGS="$(POSTSTAGE1_LDFLAGS) $(BOOT_LDFLAGS)"; export LDFLAGS; \
 	HOST_LIBS="$(POSTSTAGE1_LIBS)"; export HOST_LIBS;
@@ -828,6 +829,7 @@  POSTSTAGE1_FLAGS_TO_PASS = \
 	GNATBIND="$${GNATBIND}" \
 	LDFLAGS="$${LDFLAGS}" \
 	HOST_LIBS="$${HOST_LIBS}" \
+	$(LTO_FLAGS_TO_PASS) \
 	"`echo 'ADAFLAGS=$(BOOT_ADAFLAGS)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
 
 # Flags to pass down to makes which are built with the target environment.
diff --git a/Makefile.tpl b/Makefile.tpl
index 00dba36..f7c7e38 100644
--- a/Makefile.tpl
+++ b/Makefile.tpl
@@ -260,6 +260,7 @@  POSTSTAGE1_HOST_EXPORTS = \
 	  $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \
 	CC_FOR_BUILD="$$CC"; export CC_FOR_BUILD; \
 	$(POSTSTAGE1_CXX_EXPORT) \
+	$(LTO_EXPORTS) \
 	GNATBIND="$$r/$(HOST_SUBDIR)/prev-gcc/gnatbind"; export GNATBIND; \
 	LDFLAGS="$(POSTSTAGE1_LDFLAGS) $(BOOT_LDFLAGS)"; export LDFLAGS; \
 	HOST_LIBS="$(POSTSTAGE1_LIBS)"; export HOST_LIBS;
@@ -633,6 +634,7 @@  POSTSTAGE1_FLAGS_TO_PASS = \
 	GNATBIND="$${GNATBIND}" \
 	LDFLAGS="$${LDFLAGS}" \
 	HOST_LIBS="$${HOST_LIBS}" \
+	$(LTO_FLAGS_TO_PASS) \
 	"`echo 'ADAFLAGS=$(BOOT_ADAFLAGS)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
 
 # Flags to pass down to makes which are built with the target environment.
diff --git a/config/bootstrap-lto-slim.mk b/config/bootstrap-lto-slim.mk
new file mode 100644
index 0000000..9e065e1
--- /dev/null
+++ b/config/bootstrap-lto-slim.mk
@@ -0,0 +1,13 @@ 
+# This option enables LTO for stage2 and stage3 in slim mode
+
+STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1
+STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1
+STAGEprofile_CFLAGS += -fno-lto
+
+# assumes the host supports the linker plugin
+LTO_AR = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ar$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
+LTO_RANLIB = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ranlib$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
+
+LTO_EXPORTS = AR="$(LTO_AR)"; export AR; \
+	      RANLIB="$(LTO_RANLIB)"; export RANLIB;
+LTO_FLAGS_TO_PASS = AR="$(LTO_AR)" RANLIB="$(LTO_RANLIB)"
diff --git a/config/bootstrap-lto.mk b/config/bootstrap-lto.mk
index 27bad15..9e065e1 100644
--- a/config/bootstrap-lto.mk
+++ b/config/bootstrap-lto.mk
@@ -1,7 +1,13 @@ 
-# This option enables LTO for stage2 and stage3.
-# FIXME: Our build system is not yet able to use gcc-ar wrapper, so we need
-# to go with -ffat-lto-objects. 
+# This option enables LTO for stage2 and stage3 in slim mode
 
-STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1 -ffat-lto-objects
-STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1 -ffat-lto-objects
+STAGE2_CFLAGS += -flto=jobserver -frandom-seed=1
+STAGE3_CFLAGS += -flto=jobserver -frandom-seed=1
 STAGEprofile_CFLAGS += -fno-lto
+
+# assumes the host supports the linker plugin
+LTO_AR = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ar$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
+LTO_RANLIB = $$r/$(HOST_SUBDIR)/prev-gcc/gcc-ranlib$(exeext) -B$$r/$(HOST_SUBDIR)/prev-gcc/
+
+LTO_EXPORTS = AR="$(LTO_AR)"; export AR; \
+	      RANLIB="$(LTO_RANLIB)"; export RANLIB;
+LTO_FLAGS_TO_PASS = AR="$(LTO_AR)" RANLIB="$(LTO_RANLIB)"