Patchwork [google] Sort output of find for libtool to ensure deterministic archives

login
register
mail settings
Submitter Simon Baldwin
Date Jan. 27, 2011, 1:39 p.m.
Message ID <20110127133951.CCBD41C6CDA@hpgntab-ubiq73.eem.corp.google.com>
Download mbox | patch
Permalink /patch/80693/
State New
Headers show

Comments

Simon Baldwin - Jan. 27, 2011, 1:39 p.m.
Sort output of 'find' for libtool to ensure deterministic archives.

Strict build environments do binary comparisons of files across gcc builds.
Parallelized builds, followed by locating object files with a find command,
result in non-deterministic ordering of object files found.  The ordering
of files found can then affect archive construction, causing such strict
build environments to report differences that are false positives.

Sorting objects returned by the find command removes these false positives.

Targeted for the google/integration branch.

ChangeLog.google:
2011-01-27  Simon Baldwin  <simonb@google.com>

	* ltmain.sh (func_extract_archives): Sort the output of 'find'.

Google ref: 39145, 39106
Diego Novillo - Jan. 27, 2011, 1:42 p.m.
On Thu, Jan 27, 2011 at 08:39, Simon Baldwin <simonb@google.com> wrote:

>        * ltmain.sh (func_extract_archives): Sort the output of 'find'.

OK.


Diego.
Richard Guenther - Jan. 27, 2011, 2:13 p.m.
On Thu, Jan 27, 2011 at 2:42 PM, Diego Novillo <dnovillo@google.com> wrote:
> On Thu, Jan 27, 2011 at 08:39, Simon Baldwin <simonb@google.com> wrote:
>
>>        * ltmain.sh (func_extract_archives): Sort the output of 'find'.
>
> OK.

-> upstream?

>
> Diego.
>
Diego Novillo - Jan. 27, 2011, 2:22 p.m.
On Thu, Jan 27, 2011 at 09:13, Richard Guenther
<richard.guenther@gmail.com> wrote:
> On Thu, Jan 27, 2011 at 2:42 PM, Diego Novillo <dnovillo@google.com> wrote:
>> On Thu, Jan 27, 2011 at 08:39, Simon Baldwin <simonb@google.com> wrote:
>>
>>>        * ltmain.sh (func_extract_archives): Sort the output of 'find'.
>>
>> OK.
>
> -> upstream?

Absolutely.  We are first putting patches in google/integration that
we had doubts whether they'd be acceptable upstream.  But anything
that seems generally useful will be sent to trunk as well.

Simon, let's mark the patches that other reviewers find useful in
ChangeLog.google, I am marking this one.


Diego.
Diego Novillo - Jan. 27, 2011, 4:04 p.m.
On Thu, Jan 27, 2011 at 09:22, Diego Novillo <dnovillo@google.com> wrote:
> On Thu, Jan 27, 2011 at 09:13, Richard Guenther
> <richard.guenther@gmail.com> wrote:
>> On Thu, Jan 27, 2011 at 2:42 PM, Diego Novillo <dnovillo@google.com> wrote:
>>> On Thu, Jan 27, 2011 at 08:39, Simon Baldwin <simonb@google.com> wrote:
>>>
>>>>        * ltmain.sh (func_extract_archives): Sort the output of 'find'.
>>>
>>> OK.
>>
>> -> upstream?
>
> Absolutely.  We are first putting patches in google/integration that
> we had doubts whether they'd be acceptable upstream.  But anything
> that seems generally useful will be sent to trunk as well.
>
> Simon, let's mark the patches that other reviewers find useful in
> ChangeLog.google, I am marking this one.

Ah, you haven't committed it yet, so you get to mark it :)

I was thinking of adding a string like '[send to trunk]' right below the date:

2011-01-27  Simon Baldwin  <simonb@google.com>

       [send to trunk]
       * ltmain.sh (func_extract_archives): Sort the output of 'find'.


Diego.
Richard Guenther - Jan. 27, 2011, 4:05 p.m.
On Thu, Jan 27, 2011 at 5:04 PM, Diego Novillo <dnovillo@google.com> wrote:
> On Thu, Jan 27, 2011 at 09:22, Diego Novillo <dnovillo@google.com> wrote:
>> On Thu, Jan 27, 2011 at 09:13, Richard Guenther
>> <richard.guenther@gmail.com> wrote:
>>> On Thu, Jan 27, 2011 at 2:42 PM, Diego Novillo <dnovillo@google.com> wrote:
>>>> On Thu, Jan 27, 2011 at 08:39, Simon Baldwin <simonb@google.com> wrote:
>>>>
>>>>>        * ltmain.sh (func_extract_archives): Sort the output of 'find'.
>>>>
>>>> OK.
>>>
>>> -> upstream?
>>
>> Absolutely.  We are first putting patches in google/integration that
>> we had doubts whether they'd be acceptable upstream.  But anything
>> that seems generally useful will be sent to trunk as well.
>>
>> Simon, let's mark the patches that other reviewers find useful in
>> ChangeLog.google, I am marking this one.
>
> Ah, you haven't committed it yet, so you get to mark it :)
>
> I was thinking of adding a string like '[send to trunk]' right below the date:

send to trunk as in send to upstream libtool, trunk may be confusing.

Richard.

> 2011-01-27  Simon Baldwin  <simonb@google.com>
>
>       [send to trunk]
>       * ltmain.sh (func_extract_archives): Sort the output of 'find'.
>
>
> Diego.
>
Ollie Wild - Jan. 27, 2011, 4:55 p.m.
On Thu, Jan 27, 2011 at 8:22 AM, Diego Novillo <dnovillo@google.com> wrote:
>
> On Thu, Jan 27, 2011 at 09:13, Richard Guenther
> <richard.guenther@gmail.com> wrote:
>
>
> > -> upstream?
>
> Absolutely.  We are first putting patches in google/integration that
> we had doubts whether they'd be acceptable upstream.  But anything
> that seems generally useful will be sent to trunk as well.
>
> Simon, let's mark the patches that other reviewers find useful in
> ChangeLog.google, I am marking this one.

BTW, this has already been submitted to upstream libtool
(http://lists.gnu.org/archive/html/libtool-patches/2010-03/msg00008.html).
 That's in addition to our patch to allow --with-pic to accept package
names (http://lists.gnu.org/archive/html/libtool-patches/2010-10/msg00058.html).

Perhaps it's time to do a wholesale import of latest libtool to GCC.
Last I heard (http://lists.gnu.org/archive/html/libtool-patches/2010-11/msg00031.html),
there were some problems with --with-sysroot.

Ollie
Paolo Bonzini - Jan. 27, 2011, 5:08 p.m.
On 01/27/2011 05:55 PM, Ollie Wild wrote:
> Perhaps it's time to do a wholesale import of latest libtool to GCC.
> Last I heard (http://lists.gnu.org/archive/html/libtool-patches/2010-11/msg00031.html),
> there were some problems with --with-sysroot.

They can be worked around by disabling Libtool's --with-sysroot support.

Paolo
Ollie Wild - Jan. 27, 2011, 5:22 p.m.
On Thu, Jan 27, 2011 at 11:08 AM, Paolo Bonzini <bonzini@gnu.org> wrote:
>
> On 01/27/2011 05:55 PM, Ollie Wild wrote:
>>
>> Perhaps it's time to do a wholesale import of latest libtool to GCC.
>> Last I heard (http://lists.gnu.org/archive/html/libtool-patches/2010-11/msg00031.html),
>> there were some problems with --with-sysroot.
>
> They can be worked around by disabling Libtool's --with-sysroot support.

What are the implications of that?  No configuring GCC with --with-sysroot?

We rely heavily on that.  ;-)

Ollie
Ralf Wildenhues - Jan. 27, 2011, 5:51 p.m.
Hello,

* Ollie Wild wrote on Thu, Jan 27, 2011 at 06:22:48PM CET:
> On Thu, Jan 27, 2011 at 11:08 AM, Paolo Bonzini <bonzini@gnu.org> wrote:
> > On 01/27/2011 05:55 PM, Ollie Wild wrote:
> >> Perhaps it's time to do a wholesale import of latest libtool to GCC.
> >> Last I heard (http://lists.gnu.org/archive/html/libtool-patches/2010-11/msg00031.html),
> >> there were some problems with --with-sysroot.
> >
> > They can be worked around by disabling Libtool's --with-sysroot support.
> 
> What are the implications of that?  No configuring GCC with --with-sysroot?

No.  It turns off the new-in-Libtool-2.4 support for a --with-sysroot
flag that is not quite compatible with GCC's understanding of same flag.

I offered a wholesale update of Libtool in GCC a little while ago:
http://gcc.gnu.org/ml/gcc-patches/2011-01/msg00580.html
It was not met with enthusiasm (understandably, given the timing).

Testing such an update (including src, and smoke testing Libtool's
testsuite on 20 or so operating systems) usually takes the better part
of a weekend, which is something I cannot in earnest promise for the
next few weekends; I might be able to exchange some testing for a better
flame suite.  So *please*, if the RM's reconsider, consider deciding
before Saturday.  And yes, there were no Libtool testsuite regressions a
couple of weeks ago, but I need to redo the patches anyway for the
recent FreeBSD 1.x removal patch.

Thanks,
Ralf

Patch

Index: ltmain.sh
===================================================================
--- ltmain.sh	(revision 169330)
+++ ltmain.sh	(working copy)
@@ -2932,7 +2932,7 @@  func_extract_archives ()
         func_extract_an_archive "$my_xdir" "$my_xabs"
 	;;
       esac
-      my_oldobjs="$my_oldobjs "`find $my_xdir -name \*.$objext -print -o -name \*.lo -print | $NL2SP`
+      my_oldobjs="$my_oldobjs "`find $my_xdir -name \*.$objext -print -o -name \*.lo -print | sort | $NL2SP`
     done
 
     func_extract_archives_result="$my_oldobjs"