Patchwork [regression?] Support --static-libstdc++ with native AIX ld

login
register
mail settings
Submitter Michael Haubenwallner
Date Jan. 25, 2013, 1:55 p.m.
Message ID <51028EC7.1070807@salomon.at>
Download mbox | patch
Permalink /patch/215723/
State New
Headers show

Comments

Michael Haubenwallner - Jan. 25, 2013, 1:55 p.m.
On 01/24/2013 06:45 PM, David Edelsohn wrote:
>> Since switching to C++ inside gcc, it isn't possible to get working gcc/g++
>> binaries installed using native ld on AIX without this patch, as these need
>> dynamic libstdc++.a from builddir(!). Even after successful build with the
>> workaround from PR55105[1], installed binaries break when removing builddir.
> 
> I build GCC almost daily on AIX and test installation and do not have
> this problem.

Hmm - which "oslevel -s" do you use?

Here I've tried on 7100-01-05-1228.
Also available are 5300-08-09-1013 and 6100-07-05-1228.

According to http://www.ibm.com/developerworks/aix/library/au-aix5l-me.html
AIX does listen to LD_LIBRARY_PATH since 5.3 now.

In case you do have an AIX 5.2 or older, or some other AIX TechLevel that does
not use LD_LIBRARY_PATH for whatever reason, to trigger the same problem this
patch to toplevel configure should work I guess:


Actually, I'm wondering why this shouldn't go in anyway for consistency across platforms.

> Are you configuring GCC in a special way? Do you set the installation
> directory prefix when you configure?

I don't believe what I do is something really special.

> I build GCC with GCC and have an older version of GCC (including libstdc++) already installed.

Same here, building everything out-of-source. The prerequisites used are:
* CONFIG_SHELL=/usr/local/bin/bash 4.1.7 from bullfreeware (symlinks to /opt/freeware/bin/)
* /usr/bin/{gcc,g++} 4.6.1 from bullfreeware (symlinks to /opt/freeware/bin/)
* /usr/bin/gmake 3.82 from bullfreeware (symlinks to /opt/freeware/bin/)
* gmp-5.0.4: as shared library, configured with --prefix=/prereq ABI=32
* mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-gmp=/prereq
* mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-{gmp,mpfr}=/prereq
* gawk-3.1.7, flex-2.5.35, m4-1.4.13 from some Gentoo Prefix instance, nowhere in PATH,
  thus: export {AWK,FLEX}=/gentoo/prefix/usr/bin/{awk,flex} and this patch:
  http://gcc.gnu.org/ml/gcc-patches/2013-01/msg00960.html

For gcc:
* $CONFIG_SHELL configure --prefix=/does/not/exist/yet --with-{gmp,mpfr,mpc}=/prereq \
    --enable--languages=c,c++ --disable-werror --disable-nls
* gmake bootstrap

But unlike you, I do not use --with-boot-ldflags.

> I do not understand why you refer to native ld on AIX and what that
> has to do with any problem.

AIX ld does not understand the -Bstatic/-Bdynamic flags. According to where this very
patch adds -bstatic/-bdynamic for AIX ld, the -Bstatic/-Bdymamic flags will be used
with GNU ld for -static-libstdc++. So I'm guessing that libstdc++ will be statically
linked into gcc,g++,cc1(?) with GNU ld. However, must admit that I've never tried
hard enough to get GNU binutils working.

> $ dump -H cc1
> 
>                         ***Import File Strings***
> INDEX  PATH                          BASE                MEMBER
> 0      /tmp/20130123/prev-powerpc-ibm-aix7.1.0.0/libstdc++-v3/src/.libs:/tmp/20130123/prev-powerpc-ibm-aix7.1.0.0/libstdc++-v3/libsupc++/.libs:/usr/gnu/lib:../zlib:/tmp/20130123/./prev-gcc:/usr/lib:/lib

> Yes, it refers to the build directory, but it also include
> /usr/gnu/lib on the path and a copy of libstdc++.a is in /usr/gnu/lib.

Feels like this is because of --with-boot-ldflags=-L/usr/gnu/lib only, or because
of gmp,mpfr,mpc being found there.

> I often re-link the executable and add -Wl,-blibpath: to set a
> narrower search path.

Do you do run these re-link steps manually?

> With your patch do you configure with any additional options to cause
> the libraries to be linked statically?

Nope.

> The values you set in the patch are correct, but something still does
> not make sense with the problem.

All things considered, simply feels like your AIX doesn't listen to LD_LIBRARY_PATH.

/haubi/
David Edelsohn - Jan. 25, 2013, 3:56 p.m.
On Fri, Jan 25, 2013 at 8:55 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:

> Hmm - which "oslevel -s" do you use?
>
> Here I've tried on 7100-01-05-1228.
> Also available are 5300-08-09-1013 and 6100-07-05-1228.

I used to bootstrap on AIX 5.3.  I now bootstrap on AIX 7.1 and I try
AIX 6.1 occasionally.

> In case you do have an AIX 5.2 or older, or some other AIX TechLevel that does
> not use LD_LIBRARY_PATH for whatever reason, to trigger the same problem this
> patch to toplevel configure should work I guess:
>
> --- a/configure
> +++ b/configure
> @@ -6978,6 +6978,7 @@ rm -f conftest*
>
>  # Decide which environment variable is used to find dynamic libraries.
>  case "${host}" in
> +  *-*-aix*) RPATH_ENVVAR=LIBPATH ;;
>    *-*-hpux*) RPATH_ENVVAR=SHLIB_PATH ;;
>    *-*-darwin*) RPATH_ENVVAR=DYLD_LIBRARY_PATH ;;
>    *-*-mingw* | *-*-cygwin ) RPATH_ENVVAR=PATH ;;
>
> Actually, I'm wondering why this shouldn't go in anyway for consistency across platforms.

As you wrote, AIX 5.3 and above support LD_LIBRARY_PATH. Most AIX 4
and AIX 5.1/5.2 users are not replacing their open source toolchain.

> For gcc:
> * $CONFIG_SHELL configure --prefix=/does/not/exist/yet --with-{gmp,mpfr,mpc}=/prereq \
>     --enable--languages=c,c++ --disable-werror --disable-nls
> * gmake bootstrap
>
> But unlike you, I do not use --with-boot-ldflags.

I will try without boot-ldflags and with your patch.

> Feels like this is because of --with-boot-ldflags=-L/usr/gnu/lib only, or because
> of gmp,mpfr,mpc being found there.
>
>> I often re-link the executable and add -Wl,-blibpath: to set a
>> narrower search path.
>
> Do you do run these re-link steps manually?

Yes, I edit the Makefile to insert -Wl,-blibpath: and re-link. I don't
build and install a production version of GCC that often.

> All things considered, simply feels like your AIX doesn't listen to LD_LIBRARY_PATH.

I think this has to do with boot-ldflags.

Again, I think the values you set in your patch are correct. I simply
want to make sure it doesn't break anything.

Thanks, David
David Edelsohn - Jan. 27, 2013, 2:16 a.m.
On Fri, Jan 25, 2013 at 8:55 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:

> Same here, building everything out-of-source. The prerequisites used are:
> * CONFIG_SHELL=/usr/local/bin/bash 4.1.7 from bullfreeware (symlinks to /opt/freeware/bin/)
> * /usr/bin/{gcc,g++} 4.6.1 from bullfreeware (symlinks to /opt/freeware/bin/)
> * /usr/bin/gmake 3.82 from bullfreeware (symlinks to /opt/freeware/bin/)
> * gmp-5.0.4: as shared library, configured with --prefix=/prereq ABI=32
> * mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-gmp=/prereq
> * mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-{gmp,mpfr}=/prereq
> * gawk-3.1.7, flex-2.5.35, m4-1.4.13 from some Gentoo Prefix instance, nowhere in PATH,
>   thus: export {AWK,FLEX}=/gentoo/prefix/usr/bin/{awk,flex} and this patch:
>   http://gcc.gnu.org/ml/gcc-patches/2013-01/msg00960.html
>
> For gcc:
> * $CONFIG_SHELL configure --prefix=/does/not/exist/yet --with-{gmp,mpfr,mpc}=/prereq \
>     --enable--languages=c,c++ --disable-werror --disable-nls
> * gmake bootstrap

I committed your patch.

By the way, NLS works if you build and install GNU libiconv (1.14) and
add --with-libiconv-prefix=/prereq to force GCC bootstrap to use GNU
libiconv instead of AIX libiconv.

Thanks, David
Michael Haubenwallner - Jan. 28, 2013, 9:07 a.m.
On 01/27/2013 03:16 AM, David Edelsohn wrote:
> On Fri, Jan 25, 2013 at 8:55 AM, Michael Haubenwallner
> <michael.haubenwallner@salomon.at> wrote:
> 
>> Same here, building everything out-of-source. The prerequisites used are:
>> * CONFIG_SHELL=/usr/local/bin/bash 4.1.7 from bullfreeware (symlinks to /opt/freeware/bin/)
>> * /usr/bin/{gcc,g++} 4.6.1 from bullfreeware (symlinks to /opt/freeware/bin/)
>> * /usr/bin/gmake 3.82 from bullfreeware (symlinks to /opt/freeware/bin/)
>> * gmp-5.0.4: as shared library, configured with --prefix=/prereq ABI=32
>> * mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-gmp=/prereq
>> * mpfr-3.1.1: as shared library, configured with --prefix=/prereq --with-{gmp,mpfr}=/prereq
>> * gawk-3.1.7, flex-2.5.35, m4-1.4.13 from some Gentoo Prefix instance, nowhere in PATH,
>>   thus: export {AWK,FLEX}=/gentoo/prefix/usr/bin/{awk,flex} and this patch:
>>   http://gcc.gnu.org/ml/gcc-patches/2013-01/msg00960.html
>>
>> For gcc:
>> * $CONFIG_SHELL configure --prefix=/does/not/exist/yet --with-{gmp,mpfr,mpc}=/prereq \
>>     --enable--languages=c,c++ --disable-werror --disable-nls
>> * gmake bootstrap
> 
> I committed your patch.

Thank you!

But still curious if you've been able to reproduce the problem,
and why you didn't encounter this problem beforehand.

> By the way, NLS works if you build and install GNU libiconv (1.14) and
> add --with-libiconv-prefix=/prereq to force GCC bootstrap to use GNU
> libiconv instead of AIX libiconv.

Yes, but (you've asked) here is this situation I don't want to configure extra deplib-prefixes
for (remember bullfreeware is listed as provider for gcc-binaries):

* bullfreeware's libiconv-1.13.1 and gettext-0.17 is installed in /opt/freeware,
* /usr/lib/libintl.a is symlinked to /opt/freeware/lib (by bullfreeware's RPM),
* /usr/lib/libiconv.a is the original AIX' one.

Now, /usr/lib/libintl.a needs /opt/freeware/lib/libiconv.a[libiconv.so.2], and it does
contain the correct RUNPATH. But subsequent binaries linking against /usr/lib/libintl.a
don't (necessarily) know about the need to add /opt/freeware/lib as RUNPATH, so these
binaries break with libiconv.so.2 not being found as member of /usr/lib/libiconv.a, because
AIX unfortunately does stop its shared-library search at the first archive filename found.

This also is the main reason for my filename-based-shared-library-versioning thing.

While this topic is related, it has different reasoning - but the result does work:
[1] http://www.perzl.org/aix/index.php?n=FAQs.FAQs#toolbox-compatibility-issue

/haubi/
David Edelsohn - Jan. 28, 2013, 3:07 p.m.
On Mon, Jan 28, 2013 at 4:07 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:

> But still curious if you've been able to reproduce the problem,
> and why you didn't encounter this problem beforehand.

As I mentioned before, because of --boot-ld-flags, with earlier libgcc
and libstdc++ installed in that directory.

> Yes, but (you've asked) here is this situation I don't want to configure extra deplib-prefixes
> for (remember bullfreeware is listed as provider for gcc-binaries):
>
> * bullfreeware's libiconv-1.13.1 and gettext-0.17 is installed in /opt/freeware,
> * /usr/lib/libintl.a is symlinked to /opt/freeware/lib (by bullfreeware's RPM),
> * /usr/lib/libiconv.a is the original AIX' one.
>
> Now, /usr/lib/libintl.a needs /opt/freeware/lib/libiconv.a[libiconv.so.2], and it does
> contain the correct RUNPATH. But subsequent binaries linking against /usr/lib/libintl.a
> don't (necessarily) know about the need to add /opt/freeware/lib as RUNPATH, so these
> binaries break with libiconv.so.2 not being found as member of /usr/lib/libiconv.a, because
> AIX unfortunately does stop its shared-library search at the first archive filename found.
>
> This also is the main reason for my filename-based-shared-library-versioning thing.

Over the weekend, I successfully tested a different way to configure
and build: all static libraries.  If you build and privately install
GMP, MPFR, MPC and LIBICONV configured as static libraries
(--enable-static --disable-shared) and install in /prereq, then,
combined with your patch to enable --static-libstdc++ --static-libgcc,
the resulting GCC only depends on AIX libc.a -- no other shared
libraries. Bull Freeware can distribute the shared versions of the
libraries for other applications, but they do not need to be GCC
dependencies.

- David
Mike Stump - Jan. 28, 2013, 9:17 p.m.
On Jan 28, 2013, at 7:07 AM, David Edelsohn <dje.gcc@gmail.com> wrote:
> Over the weekend, I successfully tested a different way to configure
> and build: all static libraries.

Yeah, I think our build instructions for the dependent libraries should say to build them statically.
David Edelsohn - Jan. 28, 2013, 9:41 p.m.
On Mon, Jan 28, 2013 at 4:17 PM, Mike Stump <mrs@mrs.kithrup.com> wrote:
> On Jan 28, 2013, at 7:07 AM, David Edelsohn <dje.gcc@gmail.com> wrote:
>> Over the weekend, I successfully tested a different way to configure
>> and build: all static libraries.
>
> Yeah, I think our build instructions for the dependent libraries should say to build them statically.

A number of GCC developers who already do this.

I previously had problem on AIX with an earlier release of GCC that
was built with Graphite because one of the dependent libraries used
C++, but GCC was not linked with libstdc++ at the time. The only way
to break the C++ dependency of the library separate from GCC was
through shared libraries.

If one can link GCC against static libraries, it definitely simplifies
things and avoids potential conflicts with multiple versions of GCC
installed.

- David
Michael Haubenwallner - Jan. 30, 2013, 11:35 a.m.
On 01/28/13 16:07, David Edelsohn wrote:
> On Mon, Jan 28, 2013 at 4:07 AM, Michael Haubenwallner
> <michael.haubenwallner@salomon.at> wrote:
> 
>> But still curious if you've been able to reproduce the problem,
>> and why you didn't encounter this problem beforehand.
> 
> As I mentioned before, because of --boot-ld-flags, with earlier libgcc
> and libstdc++ installed in that directory.

But why didn't the RPATH_ENVVAR=LD_LIBRARY_PATH break for you when libatomic
is configured (in stage3) for ppc64 because of this workflow I can see there:

1: gcc/xgcc is linked by prev-gcc/xg++
1.1: against *dynamic* libstdc++.a
1.2: from prev-powerpc-ibm-aix7.1.0.0/libstdc++-v3/src/.libs/,
1.3: which is stored as runpath.

2: For a target-library (libstdc++, libatomic, ...),
   and for each multilib-variant (32/64, pthread yes/no)
2.1: LD_LIBRARY_PATH is set to that multilib variant's libstdc++.a dir

3: So LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
   32bit gcc/xgcc to build 64bit libstdc++.a.
=> This does /not/ break, as that libstdc++.a is not there yet.

4: Also, LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
   32bit gcc/xgcc to build libatomic.a.
=> This is the one that /does/ break, as that 64bit libstdc++.a is there now.

The only situation of wich I can think of not breaking here would be:
* --boot-ldflags are first on the linkline, and
* libstdc++.a found there is static only.

Or what important thing I could have missed?

Thanks!
/haubi/
David Edelsohn - Jan. 30, 2013, 2:16 p.m.
On Wed, Jan 30, 2013 at 6:35 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:

> But why didn't the RPATH_ENVVAR=LD_LIBRARY_PATH break for you when libatomic
> is configured (in stage3) for ppc64 because of this workflow I can see there:
>
> 1: gcc/xgcc is linked by prev-gcc/xg++
> 1.1: against *dynamic* libstdc++.a
> 1.2: from prev-powerpc-ibm-aix7.1.0.0/libstdc++-v3/src/.libs/,
> 1.3: which is stored as runpath.
>
> 2: For a target-library (libstdc++, libatomic, ...),
>    and for each multilib-variant (32/64, pthread yes/no)
> 2.1: LD_LIBRARY_PATH is set to that multilib variant's libstdc++.a dir
>
> 3: So LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
>    32bit gcc/xgcc to build 64bit libstdc++.a.
> => This does /not/ break, as that libstdc++.a is not there yet.
>
> 4: Also, LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
>    32bit gcc/xgcc to build libatomic.a.
> => This is the one that /does/ break, as that 64bit libstdc++.a is there now.
>
> The only situation of wich I can think of not breaking here would be:
> * --boot-ldflags are first on the linkline, and
> * libstdc++.a found there is static only.
>
> Or what important thing I could have missed?

Originally, I was using --boot-ld-flags which included /usr/gnu/lib
first in the path, so an older version of libstdc++ was found.  Now,
after your patch, cc1plus is statically linked with libstdc++.a.

- David
Michael Haubenwallner - Jan. 30, 2013, 3:44 p.m.
On 01/30/2013 03:16 PM, David Edelsohn wrote:
> <michael.haubenwallner@salomon.at> wrote:
>> 4: Also, LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
>>    32bit gcc/xgcc to build libatomic.a.
>> => This is the one that /does/ break, as that 64bit libstdc++.a is there now.

> Originally, I was using --boot-ld-flags which included /usr/gnu/lib
> first in the path, so an older version of libstdc++ was found.

Yes, but - sorry for being nit-picky - could you find out if your
/usr/gnu/lib/libstdc++.a was a static-only archive?

> Now, after your patch, cc1plus is statically linked with libstdc++.a.

Yes, although the problem was with xgcc already (before cc1plus is executed).

/haubi/
David Edelsohn - Jan. 30, 2013, 3:56 p.m.
On Wed, Jan 30, 2013 at 10:44 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:
>
> On 01/30/2013 03:16 PM, David Edelsohn wrote:
>> <michael.haubenwallner@salomon.at> wrote:
>>> 4: Also, LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
>>>    32bit gcc/xgcc to build libatomic.a.
>>> => This is the one that /does/ break, as that 64bit libstdc++.a is there now.
>
>> Originally, I was using --boot-ld-flags which included /usr/gnu/lib
>> first in the path, so an older version of libstdc++ was found.
>
> Yes, but - sorry for being nit-picky - could you find out if your
> /usr/gnu/lib/libstdc++.a was a static-only archive?

libstdc++.a is a shared library.  32 bit version.  But it was first in
the path, so satisfying GCC cc1, cc1plus, etc. built as a 32 bit
executable.  The libstdc++ ABI has not changed, so the library provide
all of the necessary functions.

Thanks, David
Michael Haubenwallner - Jan. 30, 2013, 4:03 p.m.
On 01/30/2013 04:56 PM, David Edelsohn wrote:
> On Wed, Jan 30, 2013 at 10:44 AM, Michael Haubenwallner
> <michael.haubenwallner@salomon.at> wrote:
>>
>> On 01/30/2013 03:16 PM, David Edelsohn wrote:
>>> <michael.haubenwallner@salomon.at> wrote:
>>>> 4: Also, LD_LIBRARY_PATH points to ppc64/libstdc++-v3/.libs/ while using
>>>>    32bit gcc/xgcc to build libatomic.a.
>>>> => This is the one that /does/ break, as that 64bit libstdc++.a is there now.
>>
>>> Originally, I was using --boot-ld-flags which included /usr/gnu/lib
>>> first in the path, so an older version of libstdc++ was found.
>>
>> Yes, but - sorry for being nit-picky - could you find out if your
>> /usr/gnu/lib/libstdc++.a was a static-only archive?
> 
> libstdc++.a is a shared library.  32 bit version.  But it was first in
> the path, so satisfying GCC cc1, cc1plus, etc. built as a 32 bit
> executable.  The libstdc++ ABI has not changed, so the library provide
> all of the necessary functions.

Erm - the question is why the 64bit libstdc++ found via LD_LIBRARY_PATH (set
during libatomic build) didn't break these 32bit executables in your case.

/haubi/
David Edelsohn - Jan. 30, 2013, 4:18 p.m.
On Wed, Jan 30, 2013 at 11:03 AM, Michael Haubenwallner
<michael.haubenwallner@salomon.at> wrote:

> Erm - the question is why the 64bit libstdc++ found via LD_LIBRARY_PATH (set
> during libatomic build) didn't break these 32bit executables in your case.

I do not have the build any more, but --boot-ld-flags affects the
build logic in subtle ways because of the way that it overrides the
link arguments.

- David

Patch

--- a/configure
+++ b/configure
@@ -6978,6 +6978,7 @@  rm -f conftest*
 
 # Decide which environment variable is used to find dynamic libraries.
 case "${host}" in
+  *-*-aix*) RPATH_ENVVAR=LIBPATH ;;
   *-*-hpux*) RPATH_ENVVAR=SHLIB_PATH ;;
   *-*-darwin*) RPATH_ENVVAR=DYLD_LIBRARY_PATH ;;
   *-*-mingw* | *-*-cygwin ) RPATH_ENVVAR=PATH ;;