Mark linker Makefile rules for job server build

From: Andi Kleen <ak@linux.intel.com>

-fwhopr=jobserver requires telling GNU make that the linker (= lto)
rules support job server, so that the necessary jobserver
information is passed through.

Add + to all the executable targets that run LTO for this purpose.

This improves build performance with a -fwhopr=jobserver
bootstrap (or BUILD_CONFIG=bootstrap-lto) significantly.

I didn't change Ada because that apparently doesn't support LTO.

Passes bootstrap and test suite. Ok to commit?

gcc/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Makefile.in (xgcc, cpp, cc1, collect2, lto-wrapper, gcov,
        gcov-dump, cc1-dummy, genprog, build/gcov-iov):
        Add + to build rule.

gcc/cp/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (g++, cc1plus): Add + to build rule.

gcc/fortran/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (gfortran, f951): Add + to build rule.

gcc/java/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (xgcj, jc1, jcf-dump, jvgenmain):
        Add + to build rule.

gcc/lto/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (lto1): Add + to build rule.

gcc/objc/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (cc1obj-dummy, cc1obj): Add + to build rule.

gcc/objcp/

2010-10-04  Andi Kleen <ak@linux.intel.com>

	* Make-lang.in (cc1objplus-dummy, cc1objplus): Add + to build rule.

More plus fixes
---
 gcc/Makefile.in          |   20 ++++++++++----------
 gcc/cp/Make-lang.in      |    4 ++--
 gcc/fortran/Make-lang.in |    4 ++--
 gcc/java/Make-lang.in    |    8 ++++----
 gcc/lto/Make-lang.in     |    2 +-
 gcc/objc/Make-lang.in    |    4 ++--
 gcc/objcp/Make-lang.in   |    4 ++--
 7 files changed, 23 insertions(+), 23 deletions(-)

On 10/04/2010 03:11 PM, Andi Kleen wrote:
> From: Andi Kleen<ak@linux.intel.com>
>
> -fwhopr=jobserver requires telling GNU make that the linker (= lto)
> rules support job server, so that the necessary jobserver
> information is passed through.
>
> Add + to all the executable targets that run LTO for this purpose.
>
> This improves build performance with a -fwhopr=jobserver
> bootstrap (or BUILD_CONFIG=bootstrap-lto) significantly.
>
> I didn't change Ada because that apparently doesn't support LTO.
>
> Passes bootstrap and test suite. Ok to commit?

Ok, but maybe we want to document this in the user manual too?

Paolo

> 
> Ok, but maybe we want to document this in the user manual too?

Thanks.

It already is documented in invoke.texi:

The parent Makefile will need a @samp{+} prepended to the command recipe
for this to work. This will likely only work if @env{MAKE} is 
GNU make.

-Andi

Hello,

* Andi Kleen wrote on Mon, Oct 04, 2010 at 03:11:39PM CEST:
> -fwhopr=jobserver requires telling GNU make that the linker (= lto)
> rules support job server, so that the necessary jobserver
> information is passed through.
> 
> Add + to all the executable targets that run LTO for this purpose.

FWIW, this will break 'make -n'.  I sent a feature request for GNU make:
<http://thread.gmane.org/gmane.comp.gnu.make.bugs/5015>

With GNU make-specific code, we could probably work around it.  I'm
thinking of something along these lines:

jobserv := $(shell dry=no; for f in $(MAKEFLAGS); do \
                case $$f in \
                  (*=*|--*);; \
                  (*n*) dry=yes;; \
                esac; \
              done; \
              test "$$dry" = no && echo +)

cc1: ...
        $(jobserv)$(LINK) ...


but the open parens in the case patterns are probably not portable
enough to all shells.

Although you may argue that 'make -n' is at least mildly broken in GCC
anyway.

FWIW, the '+' feature is not GNU make-specific, only the $(MAKE) string
detection is.

> This improves build performance with a -fwhopr=jobserver
> bootstrap (or BUILD_CONFIG=bootstrap-lto) significantly.

Yep.

Cheers,
Ralf

On Mon, Oct 04, 2010 at 07:42:46PM +0200, Ralf Wildenhues wrote:
> Hello,
> 
> * Andi Kleen wrote on Mon, Oct 04, 2010 at 03:11:39PM CEST:
> > -fwhopr=jobserver requires telling GNU make that the linker (= lto)
> > rules support job server, so that the necessary jobserver
> > information is passed through.
> > 
> > Add + to all the executable targets that run LTO for this purpose.
> 
> FWIW, this will break 'make -n'.  I sent a feature request for GNU make:

Thanks, I didn't know that. 

I agree that having a command line option for make would be best.
I'm not sure why make tries so hard to not always pass jobserver
anyways. 

> Although you may argue that 'make -n' is at least mildly broken in GCC
> anyway.

I'm not even sure what it is good for?

> FWIW, the '+' feature is not GNU make-specific, only the $(MAKE) string
> detection is.

jobserver is GNU make specific I believe.

-Andi

* Andi Kleen wrote on Mon, Oct 04, 2010 at 07:46:45PM CEST:
> I agree that having a command line option for make would be best.
> I'm not sure why make tries so hard to not always pass jobserver
> anyways. 

Because you can easily mess up the jobserver's state when you use the
fd's but don't follow the protocol; it might also hang if you keep a
cookie or maybe even if you keep an fd open, haven't checked.

> On Mon, Oct 04, 2010 at 07:42:46PM +0200, Ralf Wildenhues wrote:
> > Although you may argue that 'make -n' is at least mildly broken in GCC
> > anyway.
> 
> I'm not even sure what it is good for?

To see what would happen.  It may not be desirable in GCC, but generally
it can be quite helpful.  Of course GNU make functions aren't shown, so
in order to debug GNU make-heavy makefiles it helps more to use remake.

> > FWIW, the '+' feature is not GNU make-specific, only the $(MAKE) string
> > detection is.
> 
> jobserver is GNU make specific I believe.

Yes (although IIRC some BSDs have been working on one too), sorry for
being unprecise.  Executing even in dry-run is not GNU make-specific.

Cheers,
Ralf

On 10/4/2010 7:53 PM, Ralf Wildenhues wrote:
> * Andi Kleen wrote on Mon, Oct 04, 2010 at 07:46:45PM CEST:
>> I agree that having a command line option for make would be best.
>> I'm not sure why make tries so hard to not always pass jobserver
>> anyways.
> Because you can easily mess up the jobserver's state when you use the
> fd's but don't follow the protocol; it might also hang if you keep a
> cookie or maybe even if you keep an fd open, haven't checked.

Hmm maybe the solution would be to implement a more robust protocol
and then enable it unconditionally.

-Andi

* Andi Kleen wrote on Mon, Oct 04, 2010 at 10:37:46PM CEST:
>  On 10/4/2010 7:53 PM, Ralf Wildenhues wrote:
> >* Andi Kleen wrote on Mon, Oct 04, 2010 at 07:46:45PM CEST:
> >>I agree that having a command line option for make would be best.
> >>I'm not sure why make tries so hard to not always pass jobserver
> >>anyways.
> >Because you can easily mess up the jobserver's state when you use the
> >fd's but don't follow the protocol; it might also hang if you keep a
> >cookie or maybe even if you keep an fd open, haven't checked.
> 
> Hmm maybe the solution would be to implement a more robust protocol
> and then enable it unconditionally.

FWIW, I don't think the protocol is particularly non-robust.  It's just
that missing cookies are going to lead to problems, and closing file
descriptors not intended for arbitrary child processes is good defensive
programming in general.  However, this really should be discussed with
the GNU make maintainer.

Cheers,
Ralf

On Oct  4, 2010, Ralf Wildenhues <Ralf.Wildenhues@gmx.de> wrote:

> Hello,
> * Andi Kleen wrote on Mon, Oct 04, 2010 at 03:11:39PM CEST:
>> -fwhopr=jobserver requires telling GNU make that the linker (= lto)
>> rules support job server, so that the necessary jobserver
>> information is passed through.
>> 
>> Add + to all the executable targets that run LTO for this purpose.

> FWIW, this will break 'make -n'.

Would a “# $(MAKE)” comment at the end of the link command break it too?
IIRC such comments will also cause make to pass jobserver info down.

On 10/04/2010 07:42 PM, Ralf Wildenhues wrote:
> Hello,
>
> * Andi Kleen wrote on Mon, Oct 04, 2010 at 03:11:39PM CEST:
>> -fwhopr=jobserver requires telling GNU make that the linker (= lto)
>> rules support job server, so that the necessary jobserver
>> information is passed through.
>>
>> Add + to all the executable targets that run LTO for this purpose.
>
> FWIW, this will break 'make -n'.  I sent a feature request for GNU make:
> <http://thread.gmane.org/gmane.comp.gnu.make.bugs/5015>
>
> With GNU make-specific code, we could probably work around it.  I'm
> thinking of something along these lines:
>
> jobserv := $(shell dry=no; for f in $(MAKEFLAGS); do \
>                  case $$f in \
>                    (*=*|--*);; \

Any reason to check for *=*?  Even -W or -I do not appear there.  I 
always thought this would do:

ifeq (n,$(findstring n, $(filter-out --%, $(MAKEFLAGS))))
$(warning dry-run)
else
$(warning not dry-run)
endif

(BTW, adding --no-print-directory to $(MAKEFLAGS) broke GNU make's own 
example of using findstring to test MAKEFLAGS...).

Paolo

* Alexandre Oliva wrote on Tue, Oct 05, 2010 at 08:21:25AM CEST:
> On Oct  4, 2010, Ralf Wildenhues wrote:
> > * Andi Kleen wrote on Mon, Oct 04, 2010 at 03:11:39PM CEST:
> >> Add + to all the executable targets that run LTO for this purpose.
> 
> > FWIW, this will break 'make -n'.
> 
> Would a “# $(MAKE)” comment at the end of the link command break it too?

Yep.

> IIRC such comments will also cause make to pass jobserver info down.

Yep.

Cheers,
Ralf

* Paolo Bonzini wrote on Tue, Oct 05, 2010 at 12:41:24PM CEST:
> On 10/04/2010 07:42 PM, Ralf Wildenhues wrote:
> >jobserv := $(shell dry=no; for f in $(MAKEFLAGS); do \
> >                 case $$f in \
> >                   (*=*|--*);; \
> 
> Any reason to check for *=*?

Variable assignments:

$ cat Makefile
a: ; @echo $(MAKEFLAGS) # $(MAKE)
$ make -n foo=bar
n -- foo=bar

(The above came from the Automake code which is intended for portable
make.)

> Even -W or -I do not appear there.

That's because they are not meant to be passed on to recursive makes.
Some long options go there though.

> I always thought this would do:
> 
> ifeq (n,$(findstring n, $(filter-out --%, $(MAKEFLAGS))))
> $(warning dry-run)
> else
> $(warning not dry-run)
> endif

That's really interesting, because there seems to be something special
going on:

$ cat Makefile
a: ; @echo $(findstring n, $(filter-out --%, $(MAKEFLAGS)))
$ make foo=barn
n

Yet your code above seems to get it right.

> (BTW, adding --no-print-directory to $(MAKEFLAGS) broke GNU make's
> own example of using findstring to test MAKEFLAGS...).

Yes that should be fixed.

Cheers,
Ralf