From patchwork Thu May 9 03:59:42 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Merrill X-Patchwork-Id: 242708 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "localhost", Issuer "www.qmailtoaster.com" (not verified)) by ozlabs.org (Postfix) with ESMTPS id 300B02C009E for ; Thu, 9 May 2013 13:59:57 +1000 (EST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type; q=dns; s=default; b=KZvWxu70vXUNVcayc LnZhtNm4wC8hEDPt5hUDZxKYD0xRaS1FRiABrU7ZnKTrDNjRypt4jJM0eduiFQXB tTrHAXE4pEo6+6pM5ZhxflM6NZv1oZQUUwhrckpQ/HaadSWuK5oPRlbuV5FmqKVK dMNeXR67TrnYNXasJnmFTawbPc= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type; s=default; bh=4yePhE1/1QvNvJJnbBQKSDn RiyI=; b=NV3hYANkJZqgJWbX/UCF4l9bPAnpjze7jBys/K+QuDoMPNCzICKwwgB US23ptann8j190WN6FlM93MWDI0YE0xVXfSv4QneupB4SRvc3eFwY7PgVvJL4FHT Pom7zZ3YMKp3scKV1sAv05dvqBrNQPUPrzhToH3ZnSRutQ0GSbC8= Received: (qmail 11780 invoked by alias); 9 May 2013 03:59:51 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 11770 invoked by uid 89); 9 May 2013 03:59:51 -0000 X-Spam-SWARE-Status: No, score=-7.8 required=5.0 tests=AWL, BAYES_00, KHOP_THREADED, RCVD_IN_HOSTKARMA_W, RCVD_IN_HOSTKARMA_WL, RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS, TW_BJ, TW_JC autolearn=ham version=3.3.1 Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.84/v0.84-167-ge50287c) with ESMTP; Thu, 09 May 2013 03:59:48 +0000 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id r493xkr6010264 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 8 May 2013 23:59:46 -0400 Received: from [10.3.113.33] (ovpn-113-33.phx2.redhat.com [10.3.113.33]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id r493xjYf022693; Wed, 8 May 2013 23:59:45 -0400 Message-ID: <518B1F2E.4080009@redhat.com> Date: Wed, 08 May 2013 23:59:42 -0400 From: Jason Merrill User-Agent: Mozilla/5.0 (X11; Linux i686; rv:22.0) Gecko/20100101 Thunderbird/22.0a2 MIME-Version: 1.0 To: Mike Stump CC: Alexandre Oliva , gcc-patches List Subject: Re: RFC: PATCH to avoid linking multiple front ends at once with parallel make References: <518077C0.4050703@redhat.com> <34883A22-A2E4-48A6-84FB-C97BAA203E0E@comcast.net> In-Reply-To: <34883A22-A2E4-48A6-84FB-C97BAA203E0E@comcast.net> X-Virus-Found: No On 05/01/2013 07:40 PM, Mike Stump wrote: > $ bash -c "trap 'echo remove lock' 0; true"; echo $? Thanks for the suggestion. Here's a revised patch: commit 79b52becc6a8649933f7442d46e8054f8b6abd4b Author: Jason Merrill Date: Thu Apr 25 17:54:17 2013 -0400 * Makefile.in (link-mutex, remove-link-mutex): New targets. (native, mostlyclean): Remove link mutex. * configure.ac: Handle --enable-link-mutex. * */Make-lang.in: Use link mutex. diff --git a/gcc/Makefile.in b/gcc/Makefile.in index 903125e..f8968c5 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -1737,8 +1737,8 @@ start.encap: native xgcc$(exeext) cpp$(exeext) specs \ rest.encap: lang.rest.encap # This is what is made with the host's compiler # whether making a cross compiler or not. -native: config.status auto-host.h build-@POSUB@ $(LANGUAGES) \ - $(EXTRA_PROGRAMS) $(COLLECT2) lto-wrapper$(exeext) \ +native: remove-link-mutex config.status auto-host.h build-@POSUB@ \ + $(LANGUAGES) $(EXTRA_PROGRAMS) $(COLLECT2) lto-wrapper$(exeext) \ gcc-ar$(exeext) gcc-nm$(exeext) gcc-ranlib$(exeext) ifeq ($(enable_plugin),yes) @@ -4529,6 +4529,8 @@ mostlyclean: lang.mostlyclean -rm -f gtype.state # Delete genchecksum outputs -rm -f *-checksum.c +# Delete front-end linking mutex directory + -rm -rf link-mutex # Delete all files made by compilation # that don't exist in the distribution. @@ -5335,3 +5337,20 @@ po/gcc.pot: force $(MAKE) srcextra AWK=$(AWK) $(SHELL) $(srcdir)/po/exgettext \ $(XGETTEXT) gcc $(srcdir) + +# Like LINKER, but remove the link mutex. Used for linking front ends. +ifeq (@DO_LINK_MUTEX@,true) +LLINKER = trap '$(MAKE) remove-link-mutex' 0; $(LINKER) +else +LLINKER = $(LINKER) +endif + +.PHONY: link-mutex +link-mutex: +ifeq (@DO_LINK_MUTEX@,true) + while ! mkdir link-mutex 2>/dev/null; do sleep 1; done +endif + +.PHONY: remove-link-mutex +remove-link-mutex: + -rm -rf link-mutex diff --git a/gcc/ada/gcc-interface/Make-lang.in b/gcc/ada/gcc-interface/Make-lang.in index ef12b4b..c7eda7e 100644 --- a/gcc/ada/gcc-interface/Make-lang.in +++ b/gcc/ada/gcc-interface/Make-lang.in @@ -185,6 +185,7 @@ endif GCC_LINKERFLAGS = $(filter-out -Werror, $(ALL_LINKERFLAGS)) GCC_LINK=$(LINKER) $(GCC_LINKERFLAGS) $(LDFLAGS) +GCC_LLINK=$(LLINKER) $(GCC_LINKERFLAGS) $(LDFLAGS) # Lists of files for various purposes. @@ -562,7 +563,9 @@ TARGET_ADA_SRCS = # Since the RTL should be built with the latest compiler, remove the # stamp target in the parent directory whenever gnat1 is rebuilt gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS) - +$(GCC_LINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS) + $(MAKE) link-mutex + +$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \ + libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS) $(RM) stamp-gnatlib2-rts stamp-tools gnatbind$(exeext): ada/b_gnatb.o $(CONFIG_H) $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBDEPS) diff --git a/gcc/c/Make-lang.in b/gcc/c/Make-lang.in index 8310e0a..1b6189a 100644 --- a/gcc/c/Make-lang.in +++ b/gcc/c/Make-lang.in @@ -75,7 +75,8 @@ cc1-checksum.c : build/genchecksum$(build_exeext) checksum-options \ cc1-checksum.o : cc1-checksum.c $(CONFIG_H) $(SYSTEM_H) cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS) - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \ cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS) # # Build hooks: diff --git a/gcc/configure b/gcc/configure index 5161ae4..bd81007 100755 --- a/gcc/configure +++ b/gcc/configure @@ -670,6 +670,7 @@ subdirs dollar gcc_tooldir enable_lto +DO_LINK_MUTEX MAINT zlibinc zlibdir @@ -916,6 +917,7 @@ with_long_double_128 with_gc with_system_zlib enable_maintainer_mode +enable_link_mutex enable_version_specific_runtime_libs enable_plugin enable_libquadmath_support @@ -1627,6 +1629,8 @@ Optional Features: --enable-maintainer-mode enable make rules and dependencies not useful (and sometimes confusing) to the casual installer + --enable-link-mutex avoid linking multiple front-ends at once to avoid + thrashing on the build machine --enable-version-specific-runtime-libs specify that runtime libraries should be installed in a compiler-specific directory @@ -17830,7 +17834,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 17833 "configure" +#line 17837 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -17936,7 +17940,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 17939 "configure" +#line 17943 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -27044,6 +27048,26 @@ else MAINT='#' fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to avoid linking multiple front-ends at once" >&5 +$as_echo_n "checking whether to avoid linking multiple front-ends at once... " >&6; } + # Check whether --enable-link-mutex was given. +if test "${enable_link_mutex+set}" = set; then : + enableval=$enable_link_mutex; do_link_mutex=$enableval +else + do_link_mutex=no +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_mutex" >&5 +$as_echo "$do_link_mutex" >&6; } + +if test "$do_link_mutex" = "yes"; then + DO_LINK_MUTEX=true +else + DO_LINK_MUTEX=false +fi + + # -------------- # Language hooks # -------------- diff --git a/gcc/configure.ac b/gcc/configure.ac index b042f00..9ab03ec 100644 --- a/gcc/configure.ac +++ b/gcc/configure.ac @@ -4941,6 +4941,24 @@ else fi AC_SUBST(MAINT)dnl +dnl Whether to prevent multiple front-ends from linking at the same time + +AC_MSG_CHECKING([whether to avoid linking multiple front-ends at once]) + AC_ARG_ENABLE(link-mutex, +[AS_HELP_STRING([--enable-link-mutex], + [avoid linking multiple front-ends at once to avoid thrashing + on the build machine])], + do_link_mutex=$enableval, + do_link_mutex=no) +AC_MSG_RESULT($do_link_mutex) + +if test "$do_link_mutex" = "yes"; then + DO_LINK_MUTEX=true +else + DO_LINK_MUTEX=false +fi +AC_SUBST(DO_LINK_MUTEX) + # -------------- # Language hooks # -------------- diff --git a/gcc/cp/Make-lang.in b/gcc/cp/Make-lang.in index cda4897..f8b3583 100644 --- a/gcc/cp/Make-lang.in +++ b/gcc/cp/Make-lang.in @@ -100,7 +100,8 @@ cc1plus-checksum.c : build/genchecksum$(build_exeext) checksum-options \ cc1plus-checksum.o : cc1plus-checksum.c $(CONFIG_H) $(SYSTEM_H) cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS) ifeq ($(ENABLE_MAINTAINER_RULES), true) diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi index 1265129..7d7a5c7 100644 --- a/gcc/doc/install.texi +++ b/gcc/doc/install.texi @@ -1293,6 +1293,13 @@ opposite effect. If neither option is specified, the configure script will try to guess whether the @code{.init_array} and @code{.fini_array} sections are supported and, if they are, use them. +@item --enable-link-mutex +When building GCC, use a mutex to avoid linking the compilers for +multiple languages at the same time, to avoid thrashing on build +systems with limited free memory. This functionality requires that +the default shell be bash, as it uses the 'trap' built-in command. +The default is not to use such a mutex. + @item --enable-maintainer-mode The build rules that regenerate the Autoconf and Automake output files as well as the GCC master message catalog @file{gcc.pot} are normally diff --git a/gcc/fortran/Make-lang.in b/gcc/fortran/Make-lang.in index c3f826c..4e414e2 100644 --- a/gcc/fortran/Make-lang.in +++ b/gcc/fortran/Make-lang.in @@ -98,7 +98,8 @@ gfortran-cross$(exeext): gfortran$(exeext) # The compiler itself is called f951. f951$(exeext): $(F95_OBJS) \ $(BACKEND) $(LIBDEPS) attribs.o - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \ $(BACKENDLIBS) diff --git a/gcc/go/Make-lang.in b/gcc/go/Make-lang.in index dd98080..93ed920 100644 --- a/gcc/go/Make-lang.in +++ b/gcc/go/Make-lang.in @@ -76,7 +76,8 @@ GO_OBJS = \ go/unsafe.o go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) - +$(CXX) $(ALL_CXXFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS) # Documentation. diff --git a/gcc/java/Make-lang.in b/gcc/java/Make-lang.in index 7ed3220..4d2ef8a 100644 --- a/gcc/java/Make-lang.in +++ b/gcc/java/Make-lang.in @@ -99,7 +99,8 @@ jvspec.o-warn = -Wno-error jc1$(exeext): $(JAVA_OBJS) $(BACKEND) $(LIBDEPS) attribs.o rm -f $@ - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(JAVA_OBJS) $(BACKEND) $(ZLIB) $(LIBICONV) $(LIBS) attribs.o $(BACKENDLIBS) jcf-dump$(exeext): $(JCFDUMP_OBJS) $(LIBDEPS) diff --git a/gcc/lto/Make-lang.in b/gcc/lto/Make-lang.in index a0bdc26..e0db780 100644 --- a/gcc/lto/Make-lang.in +++ b/gcc/lto/Make-lang.in @@ -71,7 +71,8 @@ lto.stagefeedback: lto-warn = $(STRICT_WARN) $(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS) # Dependencies diff --git a/gcc/objc/Make-lang.in b/gcc/objc/Make-lang.in index 9e005c1..5faf54c 100644 --- a/gcc/objc/Make-lang.in +++ b/gcc/objc/Make-lang.in @@ -68,7 +68,8 @@ cc1obj-checksum.c : build/genchecksum$(build_exeext) checksum-options \ cc1obj-checksum.o : cc1obj-checksum.c $(CONFIG_H) $(SYSTEM_H) cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS) - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \ $(BACKEND) $(LIBS) $(BACKENDLIBS) diff --git a/gcc/objcp/Make-lang.in b/gcc/objcp/Make-lang.in index 731ca9e..665e91e 100644 --- a/gcc/objcp/Make-lang.in +++ b/gcc/objcp/Make-lang.in @@ -72,7 +72,8 @@ cc1objplus-checksum.c : build/genchecksum$(build_exeext) checksum-options \ cc1objplus-checksum.o : cc1objplus-checksum.c $(CONFIG_H) $(SYSTEM_H) cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS) - +$(LINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ + $(MAKE) link-mutex + +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \ $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS) # Objective C++ language specific files.