[OpenWrt-Devel,RESEND,tools] toolchain: ensure tools are built and staged before preparing toolchain

On Sat, Dec 13, 2014 at 4:28 AM, John Szakmeister <john@szakmeister.net> wrote:
[snip]
>> This doesn't look right to me, I don't think we should add the
>> toolchain/prepare step as an intermediate target for the regular build
>> process.
>> How about just adding this line to Makefile and leaving out the rest:
>>
>> toolchain/prepare: $(tools/stamp-install)
>
> Who will depend on the toolchain/prepare to make sure that it
> happens before toolchain/install?  I couldn't find this
> happening automatically in the Makefiles, that's why I went
> through the work of setting up the intermediate target.
>
> Trying your version locally fails.

Turns out this second version doesn't work either.  One of my
branches had the first version of the fix in it.  FWIW, the
first thing I tried was the following, but it didn't seem like
the right approach--though it does work correctly:


commit 161c49cdbc0b81d3e6072dc788cb5432760e7777
Author: John Szakmeister <john@szakmeister.net>
Date:   Wed Sep 17 06:47:58 2014 -0400

    toolchain: ensure tools are built and staged before preparing toolchain

    This fixes an issue where the toolchain/prepare step could run, but some
    of the necessary host tools might be missing.

    Signed-off-by: John Szakmeister <john@szakmeister.net>

On 2014-12-13 10:43, John Szakmeister wrote:
> On Sat, Dec 13, 2014 at 4:28 AM, John Szakmeister <john@szakmeister.net> wrote:
> [snip]
>>> This doesn't look right to me, I don't think we should add the
>>> toolchain/prepare step as an intermediate target for the regular build
>>> process.
>>> How about just adding this line to Makefile and leaving out the rest:
>>>
>>> toolchain/prepare: $(tools/stamp-install)
>>
>> Who will depend on the toolchain/prepare to make sure that it
>> happens before toolchain/install?  I couldn't find this
>> happening automatically in the Makefiles, that's why I went
>> through the work of setting up the intermediate target.
>>
>> Trying your version locally fails.
> 
> Turns out this second version doesn't work either.  One of my
> branches had the first version of the fix in it.  FWIW, the
> first thing I tried was the following, but it didn't seem like
> the right approach--though it does work correctly:
How do you reproduce the failure on your system? I've been building with
-j8 and higher for a long time now without seeing anything from
toolchain/ being built before tools/

- Felix

On Sat, Dec 13, 2014 at 5:11 AM, Felix Fietkau <nbd@openwrt.org> wrote:
> On 2014-12-13 10:43, John Szakmeister wrote:
>> On Sat, Dec 13, 2014 at 4:28 AM, John Szakmeister <john@szakmeister.net> wrote:
>> [snip]
>>>> This doesn't look right to me, I don't think we should add the
>>>> toolchain/prepare step as an intermediate target for the regular build
>>>> process.
>>>> How about just adding this line to Makefile and leaving out the rest:
>>>>
>>>> toolchain/prepare: $(tools/stamp-install)
>>>
>>> Who will depend on the toolchain/prepare to make sure that it
>>> happens before toolchain/install?  I couldn't find this
>>> happening automatically in the Makefiles, that's why I went
>>> through the work of setting up the intermediate target.
>>>
>>> Trying your version locally fails.
>>
>> Turns out this second version doesn't work either.  One of my
>> branches had the first version of the fix in it.  FWIW, the
>> first thing I tried was the following, but it didn't seem like
>> the right approach--though it does work correctly:
> How do you reproduce the failure on your system? I've been building with
> -j8 and higher for a long time now without seeing anything from
> toolchain/ being built before tools/

That's exactly what I do (except I use 4 jobs), but I see it descend
into toolchain first (well, after doing some other setup), and it
breaks when trying to setup the kernel headers.  I did find it
slightly easier to reproduce with `make -j4 toolchain/install` after a
`make dirclean`:

<<<<<<<
:: make -j4 toolchain/install
Collecting package info: done
Collecting target info: done
Checking 'working-make'... ok.
Checking 'case-sensitive-fs'... ok.
Checking 'getopt'... ok.
Checking 'fileutils'... ok.
Checking 'working-gcc'... ok.
Checking 'working-g++'... ok.
Checking 'ncurses'... ok.
Checking 'zlib'... ok.
Checking 'gawk'... ok.
Checking 'unzip'... ok.
Checking 'bzip2'... ok.
Checking 'perl'... ok.
Checking '/usr/bin/python2.7'... ok.
Checking 'wget'... ok.
Checking 'git'... ok.
Checking 'gnutar'... ok.
Checking 'svn'... ok.
Checking 'libssl'... ok.
Checking 'openssl'... ok.
Checking 'gnu-find'... ok.
Checking 'getopt-extended'... ok.
Checking 'file'... ok.
Checking 'non-root'... ok.
Checking 'openssl'... ok.
Checking '/usr/include/openssl/ssl.h'... ok.
Checking 'xsltproc'... ok.
Checking 'rsync'... ok.
Checking 'xgettext'... ok.
 make[1] toolchain/install
 make[2] -C toolchain/gdb prepare
 make[2] -C toolchain/binutils prepare
 make[2] -C toolchain/gcc/minimal prepare
 make[2] -C toolchain/kernel-headers prepare
make -r toolchain/install: build failed. Please re-run make with V=s
to see what's going on
make: *** [toolchain/install] Error 1
>>>>>>>

This is in on master with no patches. :-(

-John