Patchwork ARM: Use different linker path for hardfloat ABI

login
register
mail settings
Submitter Michael Hope
Date April 4, 2012, 2:39 a.m.
Message ID <CANLjY-kc0g753QNDSO28QOFdfDBuG=yC91K49p+d874-sTHXaw@mail.gmail.com>
Download mbox | patch
Permalink /patch/150580/
State New
Headers show

Comments

Michael Hope - April 4, 2012, 2:39 a.m.
On 4 April 2012 10:56, Joseph S. Myers <joseph@codesourcery.com> wrote:
> On Tue, 3 Apr 2012, Michael Hope wrote:
>
>> +#define GLIBC_DYNAMIC_LINKER \
>> +   "%{mhard-float:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
>> +    %{mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
>> +    %{!mfloat-abi=hard:%{!mhard-float:" GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "}}"
>
> (a) -mhard-float is a .opt Alias for -mfloat-abi=hard so does not need to
> be handled in specs.

Fixed.

> (b) You need to handle compilers configured with --with-float=hard, so
> make the specs depend on the default ABI the compiler was configured with.

GCC seems to take configure time options into account when evaluating
a spec file.

Tested by building a default, --with-float=hard, and
--with-float=softfp compiler then checking the loader path for all
combinations of {,-mglibc,-mbionic,-muclibc} x
{,-mhard-float,-msoft-float,-mfloat-abi=hard,-mfloat-abi=softfp}.

> (c) Please include libc-ports on future submissions and provide both the
> GCC patch and the glibc ports patch that have been tested to work together
> to build and install the library in the given path; a patch to one
> component like this cannot sensibly be considered in isolation.  I imagine
> you'll need appropriate ARM preconfigure support to detect what ABI the
> compiler is using, much like the support for MIPS, so that the right
> shlib-versions files are used.

Agreed.

>  I try to follow all ARM glibc discussions
> on libc-ports closely, as the ARM glibc maintainer; was there a previous
> discussion of the dynamic linker naming issue there that I missed?

Steve McIntyre is driving this inside Debian.  I'll ping him on the
GLIBC support.

The tricky one is new GCC with old GLIBC.  GCC may have to do a
configure time test and fall back to /lib/ld-linux.so.3 if the hard
float loader is missing.

>  (The only previous relevant discussion that I recall is one on
> patches@eglibc.org starting at
> <http://www.eglibc.org/archives/patches/msg01017.html>, regarding how the
> dynamic linker should check that a library has the right ABI, and there
> was no real followup on that after I indicated what would seem to be the
> appropriate implementation approaches and places for subsequent
> discussion.)

The patch above changes the loader to catch a mixed installation and
reject mixing incompatible libraries.  The static linker does this
currently but it's not essential.

> I have no idea whether shlib-versions files naming a file in a
> subdirectory will work - but if not, you'd need to send a patch to
> libc-alpha to support dynamic linkers in subdirectories, with appropriate
> justification for why you are doing something different from all other
> architectures.

Understood.  For now this is just a path.  There's more infrastructure
work needed if the path includes a directory.

> (d) Existing practice for Power Architecture and MIPS at least is that
> hard-float and soft-float *don't* use different library directories /
> dynamic linkers.

The goal is to have a standard loader path for all hard float distros
and, similar to how you can have a mixed 32/64 bit installation, allow
mixed softfp/hard float installations for distros that want it.  This
is a new requirement and ARM is the first one exposed to it.  I assume
Debian would push for similar changes on MIPS and PowerPC.

Do the MIPS or PowerPC loaders detect the ABI and change the library
path based on that?  I couldn't tell from the code.

> (e) Existing practice for cases that do use different dynamic linkers is
> to use a separate library directory, not just dynamic linker name, as in
> lib32 and lib64 for MIPS or libx32 for x32; it's certainly a lot easier to
> make two sets of libraries work in parallel if you have separate library
> directories like that.

Is this required, or should it be left to the distro to choose?  Once
the loader is in control then it can account for any distro specific
features, which may be the standard /lib and /usr/lib for single ABI
distros like Fedora or /usr/lib/$tuple for multiarch distros like
Ubuntu and Debian.

> So it would seem more appropriate to define a directory libhf for ARM (meaning you need a binutils patch as well to
> handle that directory, I think)

I'd like to leave that discussion for now.  The Debian goal is to
support incompatible ABIs and, past that, incompatible architectures.
libhf is ambiguous as you could have a MIPS hard float library
installed on the same system as an ARM hard float library.

> and these different Debian-style names
> could be implemented separately in a multiarch patch if someone submits
> one that properly accounts for my review comments on previous patch
> versions (failure to produce such a fixed patch being why Debian multiarch
> directory support has not got into GCC so far).

Agreed.  Note that this loader path discussion is unrelated to
multiarch.  It came from the same people so there's a family
resemblance.

(BTW Dann, apologies for stealing your patch)

-- Michael

2012-04-03  Michael Hope  <michael.hope@linaro.org>

       * config/arm/linux-eabi.h (GLIBC_DYNAMIC_LINKER_HARD_FLOAT): Define.
       (GLIBC_DYNAMIC_LINKER): Redefine to use the hard float path.

    use the GNU/Linux version, not the generic BPABI version.  */
Joseph S. Myers - April 4, 2012, 9:06 a.m.
On Wed, 4 Apr 2012, Michael Hope wrote:

> The tricky one is new GCC with old GLIBC.  GCC may have to do a
> configure time test and fall back to /lib/ld-linux.so.3 if the hard
> float loader is missing.

I don't think that's appropriate for ABI issues.  If a different dynamic 
linker name is specified, GCC should use it unconditionally (and require 
new enough glibc or a glibc installation that was appropriately 
rearranged).

> > I have no idea whether shlib-versions files naming a file in a
> > subdirectory will work - but if not, you'd need to send a patch to
> > libc-alpha to support dynamic linkers in subdirectories, with appropriate
> > justification for why you are doing something different from all other
> > architectures.
> 
> Understood.  For now this is just a path.  There's more infrastructure
> work needed if the path includes a directory.

Formally it's just a path - but an important feature of GNU/Linux and the 
GNU toolchain is consistency between different architectures and existing 
upstream practice is that the dynamic linker is always in the same 
directory as the other associated libraries and that this has the form 
/lib<something>.  In the absence of a compelling reason, which I have not 
seen stated, to do otherwise for a single case, I think that existing 
practice should be followed with the dynamic linker being in a directory 
such as /libhf.

The "more infrastructure work needed" makes clear that you need libc-alpha 
buy-in *before* putting any patches into GCC or ports.  But maybe if you 
don't try to put the dynamic linker in a different directory from the 
other libraries, it's easier to support via existing mechanisms (setting 
slibdir differently if --enable-multiarch-directories or similar)?

> Do the MIPS or PowerPC loaders detect the ABI and change the library
> path based on that?  I couldn't tell from the code.

No, they don't detect the ABI.  Both ABIs (and, for Power, the e500v1 and 
e500v2 variants - compatible with soft-float at the function-calling level 
but with some glibc ABI differences with soft-float and with each other) 
use the same directories.

> > (e) Existing practice for cases that do use different dynamic linkers is
> > to use a separate library directory, not just dynamic linker name, as in
> > lib32 and lib64 for MIPS or libx32 for x32; it's certainly a lot easier to
> > make two sets of libraries work in parallel if you have separate library
> > directories like that.
> 
> Is this required, or should it be left to the distro to choose?  Once
> the loader is in control then it can account for any distro specific
> features, which may be the standard /lib and /usr/lib for single ABI
> distros like Fedora or /usr/lib/$tuple for multiarch distros like
> Ubuntu and Debian.

I thought Fedora used the standard upstream /lib64 on x86_64 and so would 
naturally use a standard upstream /libhf where appropriate.

> > So it would seem more appropriate to define a directory libhf for ARM (meaning you need a binutils patch as well to
> > handle that directory, I think)
> 
> I'd like to leave that discussion for now.  The Debian goal is to
> support incompatible ABIs and, past that, incompatible architectures.
> libhf is ambiguous as you could have a MIPS hard float library
> installed on the same system as an ARM hard float library.

If you want both ARM and MIPS hard-float then I'd think you want both 
big-endian and little-endian ARM hard-float - but your patch defines the 
same dynamic linker name for both of those.

Standard upstream practice supports having multiple variants that 
plausibly run on the same system at the same time, such as /lib and 
/lib64, and it seems reasonable to support hard and soft float variants 
that way via a directory such as /libhf.  The Debian-style paths are not 
the default on any other architecture and I don't think it's appropriate 
to make them the default for this particular case only.

> > and these different Debian-style names
> > could be implemented separately in a multiarch patch if someone submits
> > one that properly accounts for my review comments on previous patch
> > versions (failure to produce such a fixed patch being why Debian multiarch
> > directory support has not got into GCC so far).
> 
> Agreed.  Note that this loader path discussion is unrelated to
> multiarch.  It came from the same people so there's a family
> resemblance.

I think it's directly related, and that such a path is inappropriate by 
default; that ARM should be consistent with other architectures, and that 
if you want to support paths in such subdirectories that would be a 
separate multiarch patch series for GCC, binutils and glibc (but the 
PT_INTERP would still use /lib<whatever>/<name> without subdirectories in 
any case).
Dennis Gilmore - April 4, 2012, 12:09 p.m.
On Wed, 4 Apr 2012 09:06:05 +0000 (UTC)
"Joseph S. Myers" <joseph@codesourcery.com> wrote:

> On Wed, 4 Apr 2012, Michael Hope wrote:
> 
> > The tricky one is new GCC with old GLIBC.  GCC may have to do a
> > configure time test and fall back to /lib/ld-linux.so.3 if the hard
> > float loader is missing.
> 
> I don't think that's appropriate for ABI issues.  If a different
> dynamic linker name is specified, GCC should use it unconditionally
> (and require new enough glibc or a glibc installation that was
> appropriately rearranged).
> 
> > > I have no idea whether shlib-versions files naming a file in a
> > > subdirectory will work - but if not, you'd need to send a patch to
> > > libc-alpha to support dynamic linkers in subdirectories, with
> > > appropriate justification for why you are doing something
> > > different from all other architectures.
> > 
> > Understood.  For now this is just a path.  There's more
> > infrastructure work needed if the path includes a directory.
> 
> Formally it's just a path - but an important feature of GNU/Linux and
> the GNU toolchain is consistency between different architectures and
> existing upstream practice is that the dynamic linker is always in
> the same directory as the other associated libraries and that this
> has the form /lib<something>.  In the absence of a compelling reason,
> which I have not seen stated, to do otherwise for a single case, I
> think that existing practice should be followed with the dynamic
> linker being in a directory such as /libhf.

Consistency across architectures is why Fedora does many of the things
the way it does,  It really doesn't make much sense to me to diverge
from that.

> The "more infrastructure work needed" makes clear that you need
> libc-alpha buy-in *before* putting any patches into GCC or ports.
> But maybe if you don't try to put the dynamic linker in a different
> directory from the other libraries, it's easier to support via
> existing mechanisms (setting slibdir differently if
> --enable-multiarch-directories or similar)?
> 
> > Do the MIPS or PowerPC loaders detect the ABI and change the library
> > path based on that?  I couldn't tell from the code.
> 
> No, they don't detect the ABI.  Both ABIs (and, for Power, the e500v1
> and e500v2 variants - compatible with soft-float at the
> function-calling level but with some glibc ABI differences with
> soft-float and with each other) use the same directories.
> 
> > > (e) Existing practice for cases that do use different dynamic
> > > linkers is to use a separate library directory, not just dynamic
> > > linker name, as in lib32 and lib64 for MIPS or libx32 for x32;
> > > it's certainly a lot easier to make two sets of libraries work in
> > > parallel if you have separate library directories like that.
> > 
> > Is this required, or should it be left to the distro to choose?
> > Once the loader is in control then it can account for any distro
> > specific features, which may be the standard /lib and /usr/lib for
> > single ABI distros like Fedora or /usr/lib/$tuple for multiarch
> > distros like Ubuntu and Debian.
> 
> I thought Fedora used the standard upstream /lib64 on x86_64 and so
> would naturally use a standard upstream /libhf where appropriate.

Fedora does use /lib64 on x86_64 I would personally prefer /libhfp but
wouldn't object to /libhf  though today we have f17 about to go beta
and all of rawhide built using /lib 

Fedora also has software floating point being installed into /lib also 

> > > So it would seem more appropriate to define a directory libhf for
> > > ARM (meaning you need a binutils patch as well to handle that
> > > directory, I think)


Dennis
Michael Hope - April 4, 2012, 11:32 p.m.
On 4 April 2012 21:06, Joseph S. Myers <joseph@codesourcery.com> wrote:
> On Wed, 4 Apr 2012, Michael Hope wrote:
>
>> The tricky one is new GCC with old GLIBC.  GCC may have to do a
>> configure time test and fall back to /lib/ld-linux.so.3 if the hard
>> float loader is missing.
>
> I don't think that's appropriate for ABI issues.  If a different dynamic
> linker name is specified, GCC should use it unconditionally (and require
> new enough glibc or a glibc installation that was appropriately
> rearranged).

OK.  I want GCC 4.7.1 to use the new path.  Does this mean that
released versions of GLIBC and GCC 4.7.1 will be incompatible, or does
GLIBC pick the path up from GCC?

>> > I have no idea whether shlib-versions files naming a file in a
>> > subdirectory will work - but if not, you'd need to send a patch to
>> > libc-alpha to support dynamic linkers in subdirectories, with appropriate
>> > justification for why you are doing something different from all other
>> > architectures.
>>
>> Understood.  For now this is just a path.  There's more infrastructure
>> work needed if the path includes a directory.
>
> Formally it's just a path - but an important feature of GNU/Linux and the
> GNU toolchain is consistency between different architectures and existing
> upstream practice is that the dynamic linker is always in the same
> directory as the other associated libraries and that this has the form
> /lib<something>.  In the absence of a compelling reason, which I have not
> seen stated, to do otherwise for a single case, I think that existing
> practice should be followed with the dynamic linker being in a directory
> such as /libhf.

OK.  This matches Jakub's email.

> The "more infrastructure work needed" makes clear that you need libc-alpha
> buy-in *before* putting any patches into GCC or ports.

OK.  I'm glad we had this discussion as it had to start somewhere.
I'll do a follow up across gcc-patches, libc-alpha, and binutils.

> But maybe if you
> don't try to put the dynamic linker in a different directory from the
> other libraries, it's easier to support via existing mechanisms (setting
> slibdir differently if --enable-multiarch-directories or similar)?

OK.  /libhf may fit that better.

>> Do the MIPS or PowerPC loaders detect the ABI and change the library
>> path based on that?  I couldn't tell from the code.
>
> No, they don't detect the ABI.  Both ABIs (and, for Power, the e500v1 and
> e500v2 variants - compatible with soft-float at the function-calling level
> but with some glibc ABI differences with soft-float and with each other)
> use the same directories.

Sorry, I'm confused.  I had a poke about with MIPS and it uses
different argument registers for soft and hard float.  Soft float uses
$4 and hard float $f0.  Are there shims or similar installed by the
loader?

>> > (e) Existing practice for cases that do use different dynamic linkers is
>> > to use a separate library directory, not just dynamic linker name, as in
>> > lib32 and lib64 for MIPS or libx32 for x32; it's certainly a lot easier to
>> > make two sets of libraries work in parallel if you have separate library
>> > directories like that.
>>
>> Is this required, or should it be left to the distro to choose?  Once
>> the loader is in control then it can account for any distro specific
>> features, which may be the standard /lib and /usr/lib for single ABI
>> distros like Fedora or /usr/lib/$tuple for multiarch distros like
>> Ubuntu and Debian.
>
> I thought Fedora used the standard upstream /lib64 on x86_64 and so would
> naturally use a standard upstream /libhf where appropriate.

Good.  Dennis said the same.

>> > So it would seem more appropriate to define a directory libhf for ARM (meaning you need a binutils patch as well to
>> > handle that directory, I think)
>>
>> I'd like to leave that discussion for now.  The Debian goal is to
>> support incompatible ABIs and, past that, incompatible architectures.
>> libhf is ambiguous as you could have a MIPS hard float library
>> installed on the same system as an ARM hard float library.
>
> If you want both ARM and MIPS hard-float then I'd think you want both
> big-endian and little-endian ARM hard-float - but your patch defines the
> same dynamic linker name for both of those.

Big endian is extremely uncommon on ARM and I'd rather define it when
needed.  For strictness sake I'll change the patch to use the new path
for hard float little endian only.

So:
 * Big endian: undefined, defaults to /lib/ld-linux.so.3
 * Little endian, soft float: /lib/ld-linux.so.3
 * Little endian, hard float: /libhf/ld-linux.so.3

> Standard upstream practice supports having multiple variants that
> plausibly run on the same system at the same time, such as /lib and
> /lib64, and it seems reasonable to support hard and soft float variants
> that way via a directory such as /libhf.  The Debian-style paths are not
> the default on any other architecture and I don't think it's appropriate
> to make them the default for this particular case only.

OK.  Debian multiarch covers libraries and headers but not
executables.  As a MIPS hard float /usr/bin/ls would collide with an
ARM hard float /usr/bin/ls then it's fine for the loader names to
potentially collide as well.

In practice they wouldn't as most architecture has a subtily different
loader name (cf. ld.so.1 for MIPS, ld-linux.so.2 for i386, and
ld-linux.so.3 for ARM).

>> > and these different Debian-style names
>> > could be implemented separately in a multiarch patch if someone submits
>> > one that properly accounts for my review comments on previous patch
>> > versions (failure to produce such a fixed patch being why Debian multiarch
>> > directory support has not got into GCC so far).
>>
>> Agreed.  Note that this loader path discussion is unrelated to
>> multiarch.  It came from the same people so there's a family
>> resemblance.
>
> I think it's directly related, and that such a path is inappropriate by
> default; that ARM should be consistent with other architectures, and that
> if you want to support paths in such subdirectories that would be a
> separate multiarch patch series for GCC, binutils and glibc (but the
> PT_INTERP would still use /lib<whatever>/<name> without subdirectories in
> any case).

OK.  I'll ping the Debian guys to see if there's an advantage that I'm missing.

-- Michael
Joseph S. Myers - April 5, 2012, 12:07 a.m.
On Thu, 5 Apr 2012, Michael Hope wrote:

> > I don't think that's appropriate for ABI issues.  If a different dynamic
> > linker name is specified, GCC should use it unconditionally (and require
> > new enough glibc or a glibc installation that was appropriately
> > rearranged).
> 
> OK.  I want GCC 4.7.1 to use the new path.  Does this mean that
> released versions of GLIBC and GCC 4.7.1 will be incompatible, or does
> GLIBC pick the path up from GCC?

Released versions would be incompatible (you could make GCC check at 
configure time for too-old glibc if --with-float=hard); the path needs 
hardcoding in both places.

> >> Do the MIPS or PowerPC loaders detect the ABI and change the library
> >> path based on that?  I couldn't tell from the code.
> >
> > No, they don't detect the ABI.  Both ABIs (and, for Power, the e500v1 and
> > e500v2 variants - compatible with soft-float at the function-calling level
> > but with some glibc ABI differences with soft-float and with each other)
> > use the same directories.
> 
> Sorry, I'm confused.  I had a poke about with MIPS and it uses
> different argument registers for soft and hard float.  Soft float uses
> $4 and hard float $f0.  Are there shims or similar installed by the
> loader?

No.  A system is either purely hard-float or purely soft-float, and the 
same paths are used for both so they can't coexist.  (Mismatches at 
*static* link time are detected through object attributes.)

> Big endian is extremely uncommon on ARM and I'd rather define it when
> needed.  For strictness sake I'll change the patch to use the new path
> for hard float little endian only.

I don't think that's correct; the new path should be used independent of 
endian, just as the existing path is.  But any multiarch support patch 
should presumably define separate multiarch paths for each endianness.
Michael Hope - April 5, 2012, 1:16 a.m.
On 5 April 2012 12:07, Joseph S. Myers <joseph@codesourcery.com> wrote:
> On Thu, 5 Apr 2012, Michael Hope wrote:
>
>> > I don't think that's appropriate for ABI issues.  If a different dynamic
>> > linker name is specified, GCC should use it unconditionally (and require
>> > new enough glibc or a glibc installation that was appropriately
>> > rearranged).
>>
>> OK.  I want GCC 4.7.1 to use the new path.  Does this mean that
>> released versions of GLIBC and GCC 4.7.1 will be incompatible, or does
>> GLIBC pick the path up from GCC?
>
> Released versions would be incompatible (you could make GCC check at
> configure time for too-old glibc if --with-float=hard); the path needs
> hardcoding in both places.
>
>> >> Do the MIPS or PowerPC loaders detect the ABI and change the library
>> >> path based on that?  I couldn't tell from the code.
>> >
>> > No, they don't detect the ABI.  Both ABIs (and, for Power, the e500v1 and
>> > e500v2 variants - compatible with soft-float at the function-calling level
>> > but with some glibc ABI differences with soft-float and with each other)
>> > use the same directories.
>>
>> Sorry, I'm confused.  I had a poke about with MIPS and it uses
>> different argument registers for soft and hard float.  Soft float uses
>> $4 and hard float $f0.  Are there shims or similar installed by the
>> loader?
>
> No.  A system is either purely hard-float or purely soft-float, and the
> same paths are used for both so they can't coexist.  (Mismatches at
> *static* link time are detected through object attributes.)

Ah, the same as ARM then.  The MIPS community would need something
similar to this patch if they wanted to support soft and hard float
side by side.

>> Big endian is extremely uncommon on ARM and I'd rather define it when
>> needed.  For strictness sake I'll change the patch to use the new path
>> for hard float little endian only.
>
> I don't think that's correct; the new path should be used independent of
> endian, just as the existing path is.

OK.

> But any multiarch support patch
> should presumably define separate multiarch paths for each endianness.

That's up to Debian.  I've asked for documentation on the final tuples
and what they mean as the one at:
 http://wiki.debian.org/Multiarch/Tuples

is out of date.  I prefer defining what is needed now and doing others
as needed.

-- Michael
dann frazier - April 5, 2012, 1:32 a.m.
On Wed, Apr 04, 2012 at 02:39:58PM +1200, Michael Hope wrote:
> On 4 April 2012 10:56, Joseph S. Myers <joseph@codesourcery.com> wrote:
> > On Tue, 3 Apr 2012, Michael Hope wrote:
> >
> >> +#define GLIBC_DYNAMIC_LINKER \
> >> +   "%{mhard-float:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
> >> +    %{mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
> >> +    %{!mfloat-abi=hard:%{!mhard-float:" GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "}}"
> >
> > (a) -mhard-float is a .opt Alias for -mfloat-abi=hard so does not need to
> > be handled in specs.
> 
> Fixed.
> 
> > (b) You need to handle compilers configured with --with-float=hard, so
> > make the specs depend on the default ABI the compiler was configured with.
> 
> GCC seems to take configure time options into account when evaluating
> a spec file.
> 
> Tested by building a default, --with-float=hard, and
> --with-float=softfp compiler then checking the loader path for all
> combinations of {,-mglibc,-mbionic,-muclibc} x
> {,-mhard-float,-msoft-float,-mfloat-abi=hard,-mfloat-abi=softfp}.
> 
> > (c) Please include libc-ports on future submissions and provide both the
> > GCC patch and the glibc ports patch that have been tested to work together
> > to build and install the library in the given path; a patch to one
> > component like this cannot sensibly be considered in isolation.  I imagine
> > you'll need appropriate ARM preconfigure support to detect what ABI the
> > compiler is using, much like the support for MIPS, so that the right
> > shlib-versions files are used.
> 
> Agreed.
> 
> >  I try to follow all ARM glibc discussions
> > on libc-ports closely, as the ARM glibc maintainer; was there a previous
> > discussion of the dynamic linker naming issue there that I missed?
> 
> Steve McIntyre is driving this inside Debian.  I'll ping him on the
> GLIBC support.
> 
> The tricky one is new GCC with old GLIBC.  GCC may have to do a
> configure time test and fall back to /lib/ld-linux.so.3 if the hard
> float loader is missing.
> 
> >  (The only previous relevant discussion that I recall is one on
> > patches@eglibc.org starting at
> > <http://www.eglibc.org/archives/patches/msg01017.html>, regarding how the
> > dynamic linker should check that a library has the right ABI, and there
> > was no real followup on that after I indicated what would seem to be the
> > appropriate implementation approaches and places for subsequent
> > discussion.)
> 
> The patch above changes the loader to catch a mixed installation and
> reject mixing incompatible libraries.  The static linker does this
> currently but it's not essential.
> 
> > I have no idea whether shlib-versions files naming a file in a
> > subdirectory will work - but if not, you'd need to send a patch to
> > libc-alpha to support dynamic linkers in subdirectories, with appropriate
> > justification for why you are doing something different from all other
> > architectures.
> 
> Understood.  For now this is just a path.  There's more infrastructure
> work needed if the path includes a directory.
> 
> > (d) Existing practice for Power Architecture and MIPS at least is that
> > hard-float and soft-float *don't* use different library directories /
> > dynamic linkers.
> 
> The goal is to have a standard loader path for all hard float distros
> and, similar to how you can have a mixed 32/64 bit installation, allow
> mixed softfp/hard float installations for distros that want it.  This
> is a new requirement and ARM is the first one exposed to it.  I assume
> Debian would push for similar changes on MIPS and PowerPC.
> 
> Do the MIPS or PowerPC loaders detect the ABI and change the library
> path based on that?  I couldn't tell from the code.
> 
> > (e) Existing practice for cases that do use different dynamic linkers is
> > to use a separate library directory, not just dynamic linker name, as in
> > lib32 and lib64 for MIPS or libx32 for x32; it's certainly a lot easier to
> > make two sets of libraries work in parallel if you have separate library
> > directories like that.
> 
> Is this required, or should it be left to the distro to choose?  Once
> the loader is in control then it can account for any distro specific
> features, which may be the standard /lib and /usr/lib for single ABI
> distros like Fedora or /usr/lib/$tuple for multiarch distros like
> Ubuntu and Debian.
> 
> > So it would seem more appropriate to define a directory libhf for ARM (meaning you need a binutils patch as well to
> > handle that directory, I think)
> 
> I'd like to leave that discussion for now.  The Debian goal is to
> support incompatible ABIs and, past that, incompatible architectures.
> libhf is ambiguous as you could have a MIPS hard float library
> installed on the same system as an ARM hard float library.
> 
> > and these different Debian-style names
> > could be implemented separately in a multiarch patch if someone submits
> > one that properly accounts for my review comments on previous patch
> > versions (failure to produce such a fixed patch being why Debian multiarch
> > directory support has not got into GCC so far).
> 
> Agreed.  Note that this loader path discussion is unrelated to
> multiarch.  It came from the same people so there's a family
> resemblance.
> 
> (BTW Dann, apologies for stealing your patch)

 s/stealing/improving upon/  :)

No worries, it is appreciated.

> -- Michael
> 
> 2012-04-03  Michael Hope  <michael.hope@linaro.org>
> 
>        * config/arm/linux-eabi.h (GLIBC_DYNAMIC_LINKER_HARD_FLOAT): Define.
>        (GLIBC_DYNAMIC_LINKER): Redefine to use the hard float path.
> 
> diff --git a/gcc/config/arm/linux-eabi.h b/gcc/config/arm/linux-eabi.h
> index 80bd825..8c9d2e7 100644
> --- a/gcc/config/arm/linux-eabi.h
> +++ b/gcc/config/arm/linux-eabi.h
> @@ -62,7 +62,11 @@
>  /* Use ld-linux.so.3 so that it will be possible to run "classic"
>     GNU/Linux binaries on an EABI system.  */
>  #undef  GLIBC_DYNAMIC_LINKER
> -#define GLIBC_DYNAMIC_LINKER "/lib/ld-linux.so.3"
> +#define GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "/lib/ld-linux.so.3"
> +#define GLIBC_DYNAMIC_LINKER_HARD_FLOAT
> "/lib/arm-linux-gnueabihf/ld-linux.so.3"
> +#define GLIBC_DYNAMIC_LINKER \
> +   "%{mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
> +    %{!mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "}"
> 
>  /* At this point, bpabi.h will have clobbered LINK_SPEC.  We want to
>     use the GNU/Linux version, not the generic BPABI version.  */
Konstantinos Margaritis - April 5, 2012, 1:30 p.m.
On Wed, 4 Apr 2012 07:09:46 -0500
Dennis Gilmore <dennis@gilmore.net.au> wrote:
> Fedora does use /lib64 on x86_64 I would personally prefer /libhfp but
> wouldn't object to /libhf  though today we have f17 about to go beta
> and all of rawhide built using /lib 

Hi Dennis,

  One potential problem that is born from the /libhf suggestion is the danger of having a new top level directory (/libhf) with only one file, the dynamic linker. AFAIU it, no distro is currently willing to move away from its existing scheme (/lib), Debian is most likely not going to, at best there could be a symlink /libhf -> /lib/<triplet>, but that is not solving the problem IMHO. What about other distros, if everyone is going to use a symlink for /libhf, then even suggesting it and relying on new top-level directories to solve the problem is the wrong approach. 

Loic suggested a -IMHO- better solution: to change the dynamic linker filename, not the dir, i.e. /lib/ld-linux-hf.so.3 (for this particular case).

My 2c.
Niels de Vos - April 5, 2012, 2:10 p.m.
On 04/05/2012 03:30 PM, Konstantinos Margaritis wrote:
> On Wed, 4 Apr 2012 07:09:46 -0500
> Dennis Gilmore<dennis@gilmore.net.au>  wrote:
>> Fedora does use /lib64 on x86_64 I would personally prefer /libhfp but
>> wouldn't object to /libhf  though today we have f17 about to go beta
>> and all of rawhide built using /lib
>
> Hi Dennis,
>
> One potential problem that is born from the /libhf suggestion is the
> danger of having a new top level directory (/libhf) with only one
> file, the dynamic linker. AFAIU it, no distro is currently willing to
> move away from its existing scheme (/lib), Debian is most likely not
> going to, at best there could be a symlink /libhf ->  /lib/<triplet>,
> but that is not solving the problem IMHO. What about other distros,
> if everyone is going to use a symlink for /libhf, then even
> suggesting it and relying on new top-level directories to solve the
> problem is the wrong approach.
>
> Loic suggested a -IMHO- better solution: to change the dynamic linker
> filename, not the dir, i.e. /lib/ld-linux-hf.so.3 (for this
> particular case).

Note that Fedora is a moving target and there already was a mentioning¹ 
of /lib/<triplet> for libraries earlier this year during a developers 
conference². I am not sure what the current status of that proposal is, 
but I think it is entirely possible that Fedora may follow the format 
Debian is using.

Cheers,
Niels


¹ see page 5 from this presentation: 
http://rvokal.fedorapeople.org/devconf2012/harald-A_streamlined_and_fully_compatible_Linux_Files.pdf

² http://fedoraproject.org/wiki/DeveloperConference2012
Mike Frysinger - April 5, 2012, 3:08 p.m.
On Thursday 05 April 2012 09:30:23 Konstantinos Margaritis wrote:
> On Wed, 4 Apr 2012 07:09:46 -0500 Dennis Gilmore wrote:
> > Fedora does use /lib64 on x86_64 I would personally prefer /libhfp but
> > wouldn't object to /libhf  though today we have f17 about to go beta
> > and all of rawhide built using /lib
> 
>   One potential problem that is born from the /libhf suggestion is the
> danger of having a new top level directory (/libhf) with only one file,
> the dynamic linker. AFAIU it, no distro is currently willing to move away
> from its existing scheme (/lib)

i don't think that's true.  on an x86_64 system, the 64bit libs are in 
/lib64/.  some distros tried to (pointlessly imo) resist and force 64bits into 
/lib/ when the native ABI was x86_64 (Gentoo included), but those are legacy 
imo, and afaik, they didn't break the ldso paths.

so in a setup that only has hardfloat binaries, you'd have all the libs in 
/libhf/, not just the ldso.

> Loic suggested a -IMHO- better solution: to change the dynamic linker
> filename, not the dir, i.e. /lib/ld-linux-hf.so.3 (for this particular
> case).

the implication in supporting both hardfloat and softfloat simultaneously is 
that you'd could have them both installed.  thus putting them both in /lib/ 
doesn't make much sense if you're still going to need /libhf/ to hold 
everything else.
-mike
Konstantinos Margaritis - April 5, 2012, 3:24 p.m.
On Thu, 5 Apr 2012 11:08:56 -0400
Mike Frysinger <vapier@gentoo.org> wrote:
> i don't think that's true.  on an x86_64 system, the 64bit libs are in 
> /lib64/.  some distros tried to (pointlessly imo) resist and force 64bits into 
> /lib/ when the native ABI was x86_64 (Gentoo included), but those are legacy 
> imo, and afaik, they didn't break the ldso paths.
> 
> so in a setup that only has hardfloat binaries, you'd have all the libs in 
> /libhf/, not just the ldso.

That's exactly my concern. If /libhf is chosen for the dymamic linker path, but it's not adopted by everyone else for libraries and other files, then at best you'd have a symlink, at worst a dir with only one file inside. 

> the implication in supporting both hardfloat and softfloat simultaneously is 
> that you'd could have them both installed.  thus putting them both in /lib/ 
> doesn't make much sense if you're still going to need /libhf/ to hold 
> everything else.

That case has only any chance of realization in a multiarch environment such as Debian/Ubuntu. The rest won't be affected at all. And the dynamic linkers -different filename of course- are the only libs that will be in /lib straight, the rest will be in /lib/<triplet>. So there is no danger of any conflict, at least not with libraries.
Mike Frysinger - April 5, 2012, 3:55 p.m.
On Thursday 05 April 2012 11:24:15 Konstantinos Margaritis wrote:
> On Thu, 5 Apr 2012 11:08:56 -0400 Mike Frysinger wrote:
> > i don't think that's true.  on an x86_64 system, the 64bit libs are in
> > /lib64/.  some distros tried to (pointlessly imo) resist and force 64bits
> > into /lib/ when the native ABI was x86_64 (Gentoo included), but those
> > are legacy imo, and afaik, they didn't break the ldso paths.
> > 
> > so in a setup that only has hardfloat binaries, you'd have all the libs
> > in /libhf/, not just the ldso.
> 
> That's exactly my concern. If /libhf is chosen for the dymamic linker path,
> but it's not adopted by everyone else for libraries and other files, then
> at best you'd have a symlink, at worst a dir with only one file inside.

if gcc declares libhf as another multilib target, then everyone else will get 
it automatically

note: i don't care about /lib/ld-linux-hf.so.3 or /lib/ld-linux.so.4 or 
/libhf/ld-linux.so.[34].  /lib/<triplet>/<ldso> is really the only one i don't 
think doesn't belong.

> > the implication in supporting both hardfloat and softfloat simultaneously
> > is that you'd could have them both installed.  thus putting them both in
> > /lib/ doesn't make much sense if you're still going to need /libhf/ to
> > hold everything else.
> 
> That case has only any chance of realization in a multiarch environment
> such as Debian/Ubuntu.

don't really know what you're talking about here.  other distros have no 
problem with handling multilib.
-mike
Steve McIntyre - April 5, 2012, 4:02 p.m.
On Thu, Apr 05, 2012 at 11:32:39AM +1200, Michael Hope wrote:
>
>So:
> * Big endian: undefined, defaults to /lib/ld-linux.so.3
> * Little endian, soft float: /lib/ld-linux.so.3
> * Little endian, hard float: /libhf/ld-linux.so.3
>
>> Standard upstream practice supports having multiple variants that
>> plausibly run on the same system at the same time, such as /lib and
>> /lib64, and it seems reasonable to support hard and soft float variants
>> that way via a directory such as /libhf.  The Debian-style paths are not
>> the default on any other architecture and I don't think it's appropriate
>> to make them the default for this particular case only.
>
>OK.  Debian multiarch covers libraries and headers but not
>executables.  As a MIPS hard float /usr/bin/ls would collide with an
>ARM hard float /usr/bin/ls then it's fine for the loader names to
>potentially collide as well.
>
>In practice they wouldn't as most architecture has a subtily different
>loader name (cf. ld.so.1 for MIPS, ld-linux.so.2 for i386, and
>ld-linux.so.3 for ARM).

Yes, thankfully. More by luck than any design.

Cheers,
Steve McIntyre - April 5, 2012, 4:04 p.m.
On Thu, Apr 05, 2012 at 01:16:27PM +1200, Michael Hope wrote:
>On 5 April 2012 12:07, Joseph S. Myers <joseph@codesourcery.com> wrote:
>>
>> No.  A system is either purely hard-float or purely soft-float, and the
>> same paths are used for both so they can't coexist.  (Mismatches at
>> *static* link time are detected through object attributes.)
>
>Ah, the same as ARM then.  The MIPS community would need something
>similar to this patch if they wanted to support soft and hard float
>side by side.

Yes, definitely.

>>> Big endian is extremely uncommon on ARM and I'd rather define it when
>>> needed.  For strictness sake I'll change the patch to use the new path
>>> for hard float little endian only.
>>
>> I don't think that's correct; the new path should be used independent of
>> endian, just as the existing path is.
>
>OK.
>
>> But any multiarch support patch
>> should presumably define separate multiarch paths for each endianness.
>
>That's up to Debian.  I've asked for documentation on the final tuples
>and what they mean as the one at:
> http://wiki.debian.org/Multiarch/Tuples
>
>is out of date.  I prefer defining what is needed now and doing others
>as needed.

I'm most of the way through an update for that page now; I'll ask for
comments/review shortly.

Cheers,
Steve McIntyre - April 5, 2012, 4:15 p.m.
On Thu, Apr 05, 2012 at 11:08:56AM -0400, Mike Frysinger wrote:
>On Thursday 05 April 2012 09:30:23 Konstantinos Margaritis wrote:
>> On Wed, 4 Apr 2012 07:09:46 -0500 Dennis Gilmore wrote:
>> > Fedora does use /lib64 on x86_64 I would personally prefer /libhfp but
>> > wouldn't object to /libhf  though today we have f17 about to go beta
>> > and all of rawhide built using /lib
>> 
>>   One potential problem that is born from the /libhf suggestion is the
>> danger of having a new top level directory (/libhf) with only one file,
>> the dynamic linker. AFAIU it, no distro is currently willing to move away
>> from its existing scheme (/lib)
>
>i don't think that's true.  on an x86_64 system, the 64bit libs are in 
>/lib64/.  some distros tried to (pointlessly imo) resist and force 64bits into 
>/lib/ when the native ABI was x86_64 (Gentoo included), but those are legacy 
>imo, and afaik, they didn't break the ldso paths.
>
>so in a setup that only has hardfloat binaries, you'd have all the libs in 
>/libhf/, not just the ldso.
>
>> Loic suggested a -IMHO- better solution: to change the dynamic linker
>> filename, not the dir, i.e. /lib/ld-linux-hf.so.3 (for this particular
>> case).
>
>the implication in supporting both hardfloat and softfloat simultaneously is 
>that you'd could have them both installed.  thus putting them both in /lib/ 
>doesn't make much sense if you're still going to need /libhf/ to hold 
>everything else.

Except you wouldn't - the Debian/Ubuntu plan with multi-arch is to put
them all in cleanly-separated paths corresponding to the triplets.

I'm concerned that the potential proliferation of /lib* directories
here could become very messy over time. I'm surprised that people seem
to be happy to invent another namespace on a much more ad-hoc and
arbitrary basis than the (mostly) well-understood triplets that we
already have defined in the toolchains.

Multi-arch is an attempt to make things cleaner for those of us that
care about having lots of different platforms supported in
parallel. Seen against that background, I was hoping that using the
multi-arch path for the armhf linker would make sense. For people that
don't care about multi-arch for themselves, a simple symbolic link is
all that's needed.

Cheers,
Konstantinos Margaritis - April 5, 2012, 4:25 p.m.
On Thu, 5 Apr 2012 11:55:14 -0400
Mike Frysinger <vapier@gentoo.org> wrote: 
> note: i don't care about /lib/ld-linux-hf.so.3 or /lib/ld-linux.so.4 or 
> /libhf/ld-linux.so.[34].  /lib/<triplet>/<ldso> is really the only one i don't 
> think doesn't belong.

and I'm just saying that I dislike /libhf, I also think that just raising the version is a wrong solution.

> don't really know what you're talking about here.  other distros have no 
> problem with handling multilib.

multilib for softfloat/hardfloat on arm? I don't think so, even for other arches -it was already demonstrated that you cannot e.g. have powerpc e500v2 and e600 installed concurrently, and anyway that's not the topic of the discussion here. Apart from multiarch there is no other solution to do that *for* arm, at least at the moment, because the two ABIs use exactly the same paths on a non-multiarch system. And I get back to the proposed solution /libhf -which is the multilib path you're referring to- and I'm saying that the topic here is for the linker path alone. In the hypothetical scenario that everyone agreed on /libhf for the linker path, but not for libraries -which would stay in /lib- , then we'd have a /libhf top directory with just one file, the linker. Or a symlink from /lib to /libhf or /lib/<triplet> to /libhf in Debian's case, but that defeats the purposes of having a new /libhf directory, doesn't it?

I hope I was clearer now.
Mike Frysinger - April 5, 2012, 5:37 p.m.
On Thursday 05 April 2012 12:15:41 Steve McIntyre wrote:
> On Thu, Apr 05, 2012 at 11:08:56AM -0400, Mike Frysinger wrote:
> >On Thursday 05 April 2012 09:30:23 Konstantinos Margaritis wrote:
> >> Loic suggested a -IMHO- better solution: to change the dynamic linker
> >> filename, not the dir, i.e. /lib/ld-linux-hf.so.3 (for this particular
> >> case).
> >
> >the implication in supporting both hardfloat and softfloat simultaneously
> >is that you'd could have them both installed.  thus putting them both in
> >/lib/ doesn't make much sense if you're still going to need /libhf/ to
> >hold everything else.
> 
> Except you wouldn't - the Debian/Ubuntu plan with multi-arch is to put
> them all in cleanly-separated paths corresponding to the triplets.

/lib/ and /libhf/ is just as "clean" as /lib/ and /lib64/ (and now /libx32/).  
i see no difference here with a gcc configured for these multilib paths.

> I'm concerned that the potential proliferation of /lib* directories
> here could become very messy over time. I'm surprised that people seem
> to be happy to invent another namespace on a much more ad-hoc and
> arbitrary basis than the (mostly) well-understood triplets that we
> already have defined in the toolchains.

the triplet situation isn't as clean as you imply here.  there are already 
examples of not being able to tell the ABI based purely on that.  mips64-
linux-gnu could be n32 or n64.  x86_64-linux-gnu could be x86_64 or x32.

the Debian multiarch project might have made up new triplets to account for 
this, but those don't translate exactly into the same triplet that are used 
for e.g. configure --host.

> Multi-arch is an attempt to make things cleaner for those of us that
> care about having lots of different platforms supported in
> parallel. Seen against that background, I was hoping that using the
> multi-arch path for the armhf linker would make sense.

if you think that's a useful goal, then sure.  but not everyone thinks the 
multiarch proposal is really all that useful.  however, that (much larger) 
discussion is really out of scope here.

> For people that don't care about multi-arch for themselves, a simple
> symbolic link is all that's needed.

i think it's safe to say that the wider community has yet to be convinced, so 
extending existing support via the existing standards makes more sense.
-mike
Mike Frysinger - April 10, 2012, 4:10 a.m.
On Thursday 05 April 2012 12:25:09 Konstantinos Margaritis wrote:
> On Thu, 5 Apr 2012 11:55:14 -0400 Mike Frysinger wrote:
> > note: i don't care about /lib/ld-linux-hf.so.3 or /lib/ld-linux.so.4 or
> > /libhf/ld-linux.so.[34].  /lib/<triplet>/<ldso> is really the only one i
> > don't think doesn't belong.
> 
> and I'm just saying that I dislike /libhf, I also think that just raising
> the version is a wrong solution.

i can see why bumping ver # is undesirable, but i don't think it's that big of 
a deal.  the ldso is a bit of a magic beast, so i don't think applying the 
same SONAME versioning rules is terribly important.  especially since ARM has 
already moved from ld-linux.so.2 for OABI to ld-linux.so.3 for EABI.  you 
could even argue that enshrining hardfloat is actually an ABI version bump, so 
ld-linux.so.4 is appropriate.  and really, once you bump the SONAME, injecting 
substrings like "hf" are no different.

> > don't really know what you're talking about here.  other distros have no
> > problem with handling multilib.
> 
> multilib for softfloat/hardfloat on arm? I don't think so, even for other
> arches -it was already demonstrated that you cannot e.g. have powerpc
> e500v2 and e600 installed concurrently,

i'm not familiar with ppc's embedded variants, so i can't speak to these 
examples

> and anyway that's not the topic of
> the discussion here. Apart from multiarch there is no other solution to do
> that *for* arm, at least at the moment, because the two ABIs use exactly
> the same paths on a non-multiarch system.

i'm not sure why you think that.  if people actually want to do multilib with 
these, then there's nothing stopping people from doing that, once the ldso's 
are in a unique path.

> And I get back to the proposed
> solution /libhf -which is the multilib path you're referring to- and I'm
> saying that the topic here is for the linker path alone. In the
> hypothetical scenario that everyone agreed on /libhf for the linker path,
> but not for libraries -which would stay in /lib- , then we'd have a /libhf
> top directory with just one file, the linker. Or a symlink from /lib to
> /libhf or /lib/<triplet> to /libhf in Debian's case, but that defeats the
> purposes of having a new /libhf directory, doesn't it?

what Debian chooses to do with multiarch is up to Debian ... i don't think it 
should have any bearing here.
-mike

Patch

diff --git a/gcc/config/arm/linux-eabi.h b/gcc/config/arm/linux-eabi.h
index 80bd825..8c9d2e7 100644
--- a/gcc/config/arm/linux-eabi.h
+++ b/gcc/config/arm/linux-eabi.h
@@ -62,7 +62,11 @@ 
 /* Use ld-linux.so.3 so that it will be possible to run "classic"
    GNU/Linux binaries on an EABI system.  */
 #undef  GLIBC_DYNAMIC_LINKER
-#define GLIBC_DYNAMIC_LINKER "/lib/ld-linux.so.3"
+#define GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "/lib/ld-linux.so.3"
+#define GLIBC_DYNAMIC_LINKER_HARD_FLOAT
"/lib/arm-linux-gnueabihf/ld-linux.so.3"
+#define GLIBC_DYNAMIC_LINKER \
+   "%{mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_HARD_FLOAT "} \
+    %{!mfloat-abi=hard:" GLIBC_DYNAMIC_LINKER_SOFT_FLOAT "}"

 /* At this point, bpabi.h will have clobbered LINK_SPEC.  We want to