[pull,request] Pull request for branch yem-instrument-build

Hello All!

This series adds some intrumentation to the pkg-infra so that it is easier
for the autobuilders to extract meaningfull information about the reason
of failures.

It does so by:
  - logging all messages displayed with calls to MESSAGE to a time-stamped
    log file
  - log the last such mesage to a file
  - add calls to MESSAGE in the non-package steps of the external toolchain
    backend
  - add calls to MESSAGE in the non-package steps of the crosstool-NG
    toolchain backend
  - add calls to MESAGE in the non-package steps of the internal toolchain
    backend (gcc, kernel headers, and uClibc)

Although I've tested it with external and internal toolchain backends (not
crosstool-NG for now), and although it does work, I expect some polishing
to be required. For example, the date is not displayed for the file system
images steps (I suspect it's due to second expansion being needed for
images, which is not for packages).

So, take this series as a kind of RFC, rather than a final series.


The following changes since commit 10e058b0ee7fd8135fef35bcf24ae985707078c2:

  proxychains-ng: copy proxychains.conf to target dir (2013-01-16 13:24:54 +0100)

are available in the git repository at:
  git://gitorious.org/buildroot/buildroot.git yem-instrument-build

Yann E. MORIN (6):
      pkg-infra: log current message
      toolchain/external: sprinkle with some calls to MESSAGE
      toolchain/crosstool-ng: sprinkle with some calls to MESSAGE
      toolchain/gcc: sprinkle with some calls to MESSAGE
      toolchain/kernel-headers: sprinkle with some calls to MESSAGE
      toolchain/uClibc: sprinkle with some calls to MESSAGE

 package/pkg-utils.mk                             |    9 ++++++++-
 toolchain/gcc/gcc-uclibc-4.x.mk                  |   15 +++++++++++++++
 toolchain/kernel-headers/kernel-headers.mk       |    3 +++
 toolchain/toolchain-crosstool-ng/crosstool-ng.mk |    3 ++-
 toolchain/toolchain-external/ext-tool.mk         |    8 +++++---
 toolchain/uClibc/uclibc.mk                       |   12 ++++++++++++
 6 files changed, 45 insertions(+), 5 deletions(-)

Regards,
Yann E. MORIN

On 01/17/13 00:41, Yann E. MORIN wrote:
> Although I've tested it with external and internal toolchain backends (not
> crosstool-NG for now), and although it does work, I expect some polishing
> to be required. For example, the date is not displayed for the file system
> images steps (I suspect it's due to second expansion being needed for
> images, which is not for packages).

  I consider that a bug in fs/common.mk. Patch follows to make things 
more consistent with pkg-generic.

  Regards,
  Arnout

Dear Yann E. MORIN,

On Thu, 17 Jan 2013 00:41:34 +0100, Yann E. MORIN wrote:

> So, take this series as a kind of RFC, rather than a final series.
> 
> The following changes since commit 10e058b0ee7fd8135fef35bcf24ae985707078c2:
> 
>   proxychains-ng: copy proxychains.conf to target dir (2013-01-16 13:24:54 +0100)
> 
> are available in the git repository at:
>   git://gitorious.org/buildroot/buildroot.git yem-instrument-build
> 
> Yann E. MORIN (6):
>       pkg-infra: log current message
>       toolchain/external: sprinkle with some calls to MESSAGE
>       toolchain/crosstool-ng: sprinkle with some calls to MESSAGE
>       toolchain/gcc: sprinkle with some calls to MESSAGE
>       toolchain/kernel-headers: sprinkle with some calls to MESSAGE
>       toolchain/uClibc: sprinkle with some calls to MESSAGE

I think the "pkg-infra: log current message" needs a bit more
discussion (I'm generally OK with it, just needs more time to think
about it). However, the 5 followup patches that add appropriate MESSAGE
calls in various locations of the code look good to me, and they can be
committed now in order to reduce the burden of maintaining those
patches from Yann's side.

Thanks,

Thomas

Thomas, All,

On Sunday 20 January 2013 Thomas Petazzoni wrote:
> On Thu, 17 Jan 2013 00:41:34 +0100, Yann E. MORIN wrote:
> > Yann E. MORIN (6):
> >       pkg-infra: log current message
> >       toolchain/external: sprinkle with some calls to MESSAGE
> >       toolchain/crosstool-ng: sprinkle with some calls to MESSAGE
> >       toolchain/gcc: sprinkle with some calls to MESSAGE
> >       toolchain/kernel-headers: sprinkle with some calls to MESSAGE
> >       toolchain/uClibc: sprinkle with some calls to MESSAGE
> 
> I think the "pkg-infra: log current message" needs a bit more
> discussion (I'm generally OK with it, just needs more time to think
> about it).

Yes, this one I expected some talk about.

Hooking into MESSAGE to instrument the build does not seem like the
panacea. It was just a quick-and-easy way to try and have the auto-
builders get a bit more info about a failure, so that more advanced
actions could be made (eg. bug-mail the last author of a package).

> However, the 5 followup patches that add appropriate MESSAGE
> calls in various locations of the code look good to me, and they can be
> committed now in order to reduce the burden of maintaining those
> patches from Yann's side.

I will apply Arnout's eye-candy changes, and will re-submit those patches.

I'll also revert the orderiung of the patches: the new calls to MESSAGE
do not depend on patch 1/5.

Regards,
Yann E. MORIN.