Fix up postboot dependencies [PR106472]

On Wed, Mar 13, 2024 at 10:13:37AM +0100, Jakub Jelinek wrote:
> While the first Makefile.tpl hunk looks obviously ok, the others look
> completely wrong to me.
> There is nothing special about libgo vs. libbacktrace/libatomic
> compared to any other target library which is not bootstrapped vs. any
> of its dependencies which are in the bootstrapped set.
> So, Makefile.tpl shouldn't hardcode such dependencies.

Here is my version of the fix.
The dependencies in the toplevel Makefile simply didn't take into account
that some target modules could be in a bootstrapped build built in some
configurations as bootstrap modules (typically as dependencies of other
target bootstrap modules), while in other configurations just as
dependencies of non-bootstrap target modules and so not built during the
bootstrap, but after it.
Makefile.tpl arranges for those postboot target module -> target module
dependencies to be emitted only inside of an @unless gcc-bootstrap block,
while for @if gcc-bootstrap it just emits
configure-target-whatever: stage_last
dependencies which ensure those postbootstrap target modules are only built
after everything that is bootstrapped has been.

Now, the libbacktrace/libatomic target modules have bootstrap=true
target_modules = { module= libbacktrace; bootstrap=true; };
target_modules = { module= libatomic; bootstrap=true; lib_path=.libs; };
because those modules are dependencies of libphobos target module, so
when d is included among bootstrapped languages, those are all bootstrapped
and everything works correctly.
While if d is not included, libphobos target module is disabled,
libbacktrace/libatomic target modules aren't bootstrapped, nothing during
bootstrap needs them, but post bootstrap libgo target module depends on
the libatomic and libbacktrace target modules, libgfortran target module
depends on the libbacktrace target module and libgm2 target module depends
on the libatomic target module, but those dependencies were emitted only
@unless gcc-bootstrap.  There is a similar theoretical problem for zlib
target module if GCJ would be ressurected, libphobos as bootstrap target
module depends on the zlib target module, but if d is not configured,
fastjar also depends on it.

The following patch arranges for the @if gcc-bootstrap case to emit also
target module -> target module dependencies, but conditionally on the
on dependency not being bootstrapped.

In the generated Makefile.in you can see what the Makefile.tpl change
produces and that it just adds extra dependencies which weren't there
before in the @if gcc-bootstrap case.

I've bootstrapped without this patch with
../configure --enable-languages=c,c++,go; make
on x86_64-linux (note, make -j2 or higher usually worked) which failed
as described in the PR, then with this patch with the same command which
built fine and the Makefile difference between the two builds being


	Jakub

On Tue, 2 Apr 2024, Jakub Jelinek wrote:

> On Wed, Mar 13, 2024 at 10:13:37AM +0100, Jakub Jelinek wrote:
> > While the first Makefile.tpl hunk looks obviously ok, the others look
> > completely wrong to me.
> > There is nothing special about libgo vs. libbacktrace/libatomic
> > compared to any other target library which is not bootstrapped vs. any
> > of its dependencies which are in the bootstrapped set.
> > So, Makefile.tpl shouldn't hardcode such dependencies.
> 
> Here is my version of the fix.
> The dependencies in the toplevel Makefile simply didn't take into account
> that some target modules could be in a bootstrapped build built in some
> configurations as bootstrap modules (typically as dependencies of other
> target bootstrap modules), while in other configurations just as
> dependencies of non-bootstrap target modules and so not built during the
> bootstrap, but after it.
> Makefile.tpl arranges for those postboot target module -> target module
> dependencies to be emitted only inside of an @unless gcc-bootstrap block,
> while for @if gcc-bootstrap it just emits
> configure-target-whatever: stage_last
> dependencies which ensure those postbootstrap target modules are only built
> after everything that is bootstrapped has been.
> 
> Now, the libbacktrace/libatomic target modules have bootstrap=true
> target_modules = { module= libbacktrace; bootstrap=true; };
> target_modules = { module= libatomic; bootstrap=true; lib_path=.libs; };
> because those modules are dependencies of libphobos target module, so
> when d is included among bootstrapped languages, those are all bootstrapped
> and everything works correctly.
> While if d is not included, libphobos target module is disabled,
> libbacktrace/libatomic target modules aren't bootstrapped, nothing during
> bootstrap needs them, but post bootstrap libgo target module depends on
> the libatomic and libbacktrace target modules, libgfortran target module
> depends on the libbacktrace target module and libgm2 target module depends
> on the libatomic target module, but those dependencies were emitted only
> @unless gcc-bootstrap.  There is a similar theoretical problem for zlib
> target module if GCJ would be ressurected, libphobos as bootstrap target
> module depends on the zlib target module, but if d is not configured,
> fastjar also depends on it.
> 
> The following patch arranges for the @if gcc-bootstrap case to emit also
> target module -> target module dependencies, but conditionally on the
> on dependency not being bootstrapped.
> 
> In the generated Makefile.in you can see what the Makefile.tpl change
> produces and that it just adds extra dependencies which weren't there
> before in the @if gcc-bootstrap case.
> 
> I've bootstrapped without this patch with
> ../configure --enable-languages=c,c++,go; make
> on x86_64-linux (note, make -j2 or higher usually worked) which failed
> as described in the PR, then with this patch with the same command which
> built fine and the Makefile difference between the two builds being
> diff -up obj40{a,b}/Makefile
> --- obj40a/Makefile	2024-03-31 00:35:22.243791499 +0100
> +++ obj40b/Makefile	2024-03-31 22:40:38.143299144 +0200
> @@ -29376,6 +29376,14 @@ configure-bison: stage_last
>  configure-flex: stage_last
>  configure-m4: stage_last
>  
> +configure-target-fastjar: maybe-configure-target-zlib
> +all-target-fastjar: maybe-all-target-zlib
> +all-target-libgo: maybe-all-target-libbacktrace
> +all-target-libgo: maybe-all-target-libatomic
> +all-target-libgm2: maybe-all-target-libatomic
> +configure-target-libgfortran: maybe-all-target-libbacktrace
> +configure-target-libgo: maybe-all-target-libbacktrace
> +
>  
>  # Dependencies for target modules on other target modules are
>  # described by lang_env_dependencies; the defaults apply to anything
> 
> which I believe are exactly the extra dependencies we want.
> Plus I've done normal x86_64-linux and i686-linux bootstraps/regtests
> which in my case include --enable-languages=default,ada,obj-c++,lto,go,d,rust,m2
> for x86_64 and the same except ada for i686; those with my usual make -j32.
> The Makefile difference in those builds vs. unpatched case
> is just an extra empty line.
> 
> Ok for trunk?

OK.

Richard.

> 2024-04-02  Jakub Jelinek  <jakub@redhat.com>
> 
> 	PR bootstrap/106472
> 	* Makefile.tpl (make-postboot-target-dep): New lambda.
> 	Use it to add --enable-bootstrap dependencies of target modules
> 	on other target modules if the latter aren't bootstrapped.
> 	* Makefile.in: Regenerate.
> 
> --- Makefile.tpl.jj	2024-01-09 22:40:16.812824317 +0100
> +++ Makefile.tpl	2024-03-30 14:23:51.985398859 +0100
> @@ -2013,6 +2013,25 @@ configure-target-[+module+]: maybe-all-g
>  	 (unless (=* target "target-")
>             (string-append "configure-" target ": " dep "\n"))))))
>  
> +   ;; Dependencies in between target modules if the dependencies
> +   ;; are bootstrap target modules and the target modules which
> +   ;; depend on them are emitted inside of @unless gcc-bootstrap.
> +   ;; Unfortunately, some target modules like libatomic or libbacktrace
> +   ;; have bootstrap flag set, but whether they are actually built
> +   ;; during bootstrap or after bootstrap depends on e.g. enabled languages;
> +   ;; if d is enabled, libphobos is built as target module and depends
> +   ;; on libatomic and libbacktrace, which are therefore also built as
> +   ;; bootstrap modules.  If d is not enabled but go is, libatomic and
> +   ;; libbacktrace are just dependencies of libgo which is not a bootstrap
> +   ;; target module, but we need dependencies on libatomic and libbacktrace
> +   ;; in that case even when gcc-bootstrap.  This lambda emits those.
> +   (define make-postboot-target-dep (lambda ()
> +     (let ((target (dep-module "module")) (on (dep-module "on")))
> +       (when (=* on "target-")
> +	 (when (=* target "target-")
> +	   (string-append "@unless " on "-bootstrap\n" (make-dep "" "")
> +			  "\n@endunless " on "-bootstrap\n"))))))
> +
>     ;; We now build the hash table that is used by dep-kind.
>     (define boot-modules (make-hash-table 113))
>     (define postboot-targets (make-hash-table 113))
> @@ -2045,6 +2064,11 @@ configure-target-[+module+]: maybe-all-g
>  [+ == "postbootstrap" +][+ (make-postboot-dep) +][+ ESAC +][+
>  ENDFOR dependencies +]@endif gcc-bootstrap
>  
> +@if gcc-bootstrap
> +[+ FOR dependencies +][+ CASE (dep-kind) +]
> +[+ == "postbootstrap" +][+ (make-postboot-target-dep) +][+ ESAC +][+
> +ENDFOR dependencies +]@endif gcc-bootstrap
> +
>  @unless gcc-bootstrap
>  [+ FOR dependencies +][+ CASE (dep-kind) +]
>  [+ == "postbootstrap" +][+ (make-dep "" "") +]
> --- Makefile.in.jj	2024-01-16 22:51:10.233410651 +0100
> +++ Makefile.in	2024-03-29 15:50:34.676632723 +0100
> @@ -68677,6 +68677,39 @@ configure-flex: stage_last
>  configure-m4: stage_last
>  @endif gcc-bootstrap
>  
> +@if gcc-bootstrap
> +@unless target-zlib-bootstrap
> +configure-target-fastjar: maybe-configure-target-zlib
> +@endunless target-zlib-bootstrap
> +@unless target-zlib-bootstrap
> +all-target-fastjar: maybe-all-target-zlib
> +@endunless target-zlib-bootstrap
> +@unless target-libstdc++-v3-bootstrap
> +configure-target-libgo: maybe-all-target-libstdc++-v3
> +@endunless target-libstdc++-v3-bootstrap
> +@unless target-libbacktrace-bootstrap
> +all-target-libgo: maybe-all-target-libbacktrace
> +@endunless target-libbacktrace-bootstrap
> +@unless target-libatomic-bootstrap
> +all-target-libgo: maybe-all-target-libatomic
> +@endunless target-libatomic-bootstrap
> +@unless target-libstdc++-v3-bootstrap
> +configure-target-libgm2: maybe-all-target-libstdc++-v3
> +@endunless target-libstdc++-v3-bootstrap
> +@unless target-libatomic-bootstrap
> +all-target-libgm2: maybe-all-target-libatomic
> +@endunless target-libatomic-bootstrap
> +@unless target-libstdc++-v3-bootstrap
> +configure-target-libgrust: maybe-all-target-libstdc++-v3
> +@endunless target-libstdc++-v3-bootstrap
> +@unless target-libbacktrace-bootstrap
> +configure-target-libgfortran: maybe-all-target-libbacktrace
> +@endunless target-libbacktrace-bootstrap
> +@unless target-libbacktrace-bootstrap
> +configure-target-libgo: maybe-all-target-libbacktrace
> +@endunless target-libbacktrace-bootstrap
> +@endif gcc-bootstrap
> +
>  @unless gcc-bootstrap
>  all-gnattools: maybe-all-target-libstdc++-v3
>  configure-libcc1: maybe-configure-gcc
> 
> 
> 	Jakub
> 
>