Patchwork Passing arguments to the linker when external toolchain is used.

login
register
mail settings
Submitter Ronny Meeus
Date Nov. 23, 2012, 1:04 p.m.
Message ID <CAMJ=MEc1T6Uz2tLS3ybdp5hwv7zX15ufGYMVuAY7jOSA+U=sYg@mail.gmail.com>
Download mbox | patch
Permalink /patch/201313/
State Not Applicable
Headers show

Comments

Ronny Meeus - Nov. 23, 2012, 1:04 p.m.
Hello

I'm using buildroot to build an environment for MIPS (Cavium).
For this I use a 64 bit toolchain provided by Cavium as an external toolchain.

My buildroot configuration is as follows:
* Buildroot 2012.05
* Target Architecture: MIPS (big endian)
* Target Architecture Variant: mips 64
* Target ABI: n32
* glibc

In my application we need to create partially linked objects that will
be linked together in a final link step.
I see that the correct options (ABI etc) are passed to the compiler
via the ext-toolchain-wrapper construction, but for the linker this
technique is not used. It is directly invoked without modifying the
command line. If I want to create this partially linked file, I need
to pass an extra option the linker to indicate that is needs to use
the N32 abi (-melf32btsmipn32).

To make this work, I needed to make 2 changes:
- make the linker a link to the ext-toolchain-wrapper
- change the ext-toolchain-wrapper so that it passes to options to the
linker (currently the option is hardcoded but this needs to be changed
so that the BR2_TARGET_LDFLAGS are used.
Both changes are shown below.

Please comment on whether this approach is the correct one or is there
a better alternative available.

$ hg diff toolchain/toolchain-external/ext-toolchain-wrapper.c

---
Ronny
Arnout Vandecappelle - Nov. 24, 2012, 11:33 p.m.
On 23/11/12 14:04, Ronny Meeus wrote:
> Hello
>
> I'm using buildroot to build an environment for MIPS (Cavium).
> For this I use a 64 bit toolchain provided by Cavium as an external toolchain.
>
> My buildroot configuration is as follows:
> * Buildroot 2012.05
> * Target Architecture: MIPS (big endian)
> * Target Architecture Variant: mips 64
> * Target ABI: n32
> * glibc
>
> In my application we need to create partially linked objects that will
> be linked together in a final link step.
> I see that the correct options (ABI etc) are passed to the compiler
> via the ext-toolchain-wrapper construction, but for the linker this
> technique is not used. It is directly invoked without modifying the
> command line. If I want to create this partially linked file, I need
> to pass an extra option the linker to indicate that is needs to use
> the N32 abi (-melf32btsmipn32).

  First of all, I don't understand why ld needs any option that we normally
pass to the compiler.  Do the -march, -mtune and -mfpu options even work
in ld?

  That said, if you do need the CFLAGS, then perhaps you should use
'$(CC) -nostdlib' rather than $(LD) (and of course pass ld-specific
options using -Wl,...).

>
> To make this work, I needed to make 2 changes:
> - make the linker a link to the ext-toolchain-wrapper
> - change the ext-toolchain-wrapper so that it passes to options to the
> linker (currently the option is hardcoded but this needs to be changed
> so that the BR2_TARGET_LDFLAGS are used.

  Since not all gcc flags are supported by ld, I think it should be a
separate toolchain wrapper, specific for ld.  It can be generated from
the same source code, though (using ifdefs).


  Regards,
  Arnout
Ronny Meeus - Nov. 25, 2012, 5:23 p.m.
On Sun, Nov 25, 2012 at 12:33 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 23/11/12 14:04, Ronny Meeus wrote:
>>
>> Hello
>>
>> I'm using buildroot to build an environment for MIPS (Cavium).
>> For this I use a 64 bit toolchain provided by Cavium as an external
>> toolchain.
>>
>> My buildroot configuration is as follows:
>> * Buildroot 2012.05
>> * Target Architecture: MIPS (big endian)
>> * Target Architecture Variant: mips 64
>> * Target ABI: n32
>> * glibc
>>
>> In my application we need to create partially linked objects that will
>> be linked together in a final link step.
>> I see that the correct options (ABI etc) are passed to the compiler
>> via the ext-toolchain-wrapper construction, but for the linker this
>> technique is not used. It is directly invoked without modifying the
>> command line. If I want to create this partially linked file, I need
>> to pass an extra option the linker to indicate that is needs to use
>> the N32 abi (-melf32btsmipn32).
>
>
>  First of all, I don't understand why ld needs any option that we normally
> pass to the compiler.  Do the -march, -mtune and -mfpu options even work
> in ld?

I think you miss-understood my mail.
The linker needs a different option (-melf32btsmipn32) while the
compiler needs -march, -mtune and -mfpu etc.
For the compiler there is no issue, it is the linker for which no
options are passed.

If you look to the changes I made to the wrapper, you can see that a
completely different set of options can be passed to the linker. The
compiler options are passed to the compiler while a different set of
options is passed to the linker.

>  That said, if you do need the CFLAGS, then perhaps you should use
> '$(CC) -nostdlib' rather than $(LD) (and of course pass ld-specific
> options using -Wl,...).
>
>>
>> To make this work, I needed to make 2 changes:
>> - make the linker a link to the ext-toolchain-wrapper
>> - change the ext-toolchain-wrapper so that it passes to options to the
>> linker (currently the option is hardcoded but this needs to be changed
>> so that the BR2_TARGET_LDFLAGS are used.
>
>
>  Since not all gcc flags are supported by ld, I think it should be a
> separate toolchain wrapper, specific for ld.  It can be generated from
> the same source code, though (using ifdefs).
>

That is also an option. In my code I look to the name of the program
that is executed, so at runtime the wrapper determines which set of
options need to be used.

>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                               arnout at mind be
> Senior Embedded Software Architect                 +32-16-286540
> Essensium/Mind                                     http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Arnout Vandecappelle - Nov. 25, 2012, 8:44 p.m.
On 25/11/12 18:23, Ronny Meeus wrote:
>>> >>  In my application we need to create partially linked objects that will
>>> >>  be linked together in a final link step.
>>> >>  I see that the correct options (ABI etc) are passed to the compiler
>>> >>  via the ext-toolchain-wrapper construction, but for the linker this
>>> >>  technique is not used. It is directly invoked without modifying the
>>> >>  command line. If I want to create this partially linked file, I need
>>> >>  to pass an extra option the linker to indicate that is needs to use
>>> >>  the N32 abi (-melf32btsmipn32).
>> >
>> >
>> >    First of all, I don't understand why ld needs any option that we normally
>> >  pass to the compiler.  Do the -march, -mtune and -mfpu options even work
>> >  in ld?
> I think you miss-understood my mail.
> The linker needs a different option (-melf32btsmipn32) while the
> compiler needs -march, -mtune and -mfpu etc.
> For the compiler there is no issue, it is the linker for which no
> options are passed.

  In that case, why not just pass them with LDFLAGS?


> If you look to the changes I made to the wrapper, you can see that a
> completely different set of options can be passed to the linker. The
> compiler options are passed to the compiler while a different set of
> options is passed to the linker.

  Sorry, I didn't look at the patch in that much detail...


  Regards,
  Arnout
Ronny Meeus - Nov. 25, 2012, 8:56 p.m.
On Sun, Nov 25, 2012 at 9:44 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 25/11/12 18:23, Ronny Meeus wrote:
>>>>
>>>> >>  In my application we need to create partially linked objects that
>>>> >> will
>>>> >>  be linked together in a final link step.
>>>> >>  I see that the correct options (ABI etc) are passed to the compiler
>>>> >>  via the ext-toolchain-wrapper construction, but for the linker this
>>>> >>  technique is not used. It is directly invoked without modifying the
>>>> >>  command line. If I want to create this partially linked file, I need
>>>> >>  to pass an extra option the linker to indicate that is needs to use
>>>> >>  the N32 abi (-melf32btsmipn32).
>>>
>>> >
>>> >
>>> >    First of all, I don't understand why ld needs any option that we
>>> > normally
>>> >  pass to the compiler.  Do the -march, -mtune and -mfpu options even
>>> > work
>>> >  in ld?
>>
>> I think you miss-understood my mail.
>> The linker needs a different option (-melf32btsmipn32) while the
>> compiler needs -march, -mtune and -mfpu etc.
>> For the compiler there is no issue, it is the linker for which no
>> options are passed.
>
>
>  In that case, why not just pass them with LDFLAGS?
>

Our SW is build in 2 parts: the Linux kernel, buildroot and other
components are build via buildroot itself.
Next to this we also have other applications that we build in with "a
private build infrastructure" using the toolchain, include files etc
of buildroot.

By adding the extra linker parameters with the wrapper construction we
can transparantly introduce a new toolchain (with the required
parameters) in that environment without impact on our build
infrastructure. The big advantage is that the complete toolchain
selection (CPU,ABI etc) only needs to be done in buildroot.

>
>> If you look to the changes I made to the wrapper, you can see that a
>> completely different set of options can be passed to the linker. The
>> compiler options are passed to the compiler while a different set of
>> options is passed to the linker.
>
>
>  Sorry, I didn't look at the patch in that much detail...
>
>
>
>  Regards,
>  Arnout
> --
> Arnout Vandecappelle                               arnout at mind be
> Senior Embedded Software Architect                 +32-16-286540
> Essensium/Mind                                     http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Arnout Vandecappelle - Nov. 25, 2012, 10:25 p.m.
On 25/11/12 21:56, Ronny Meeus wrote:
> On Sun, Nov 25, 2012 at 9:44 PM, Arnout Vandecappelle<arnout@mind.be>  wrote:
[snip]
>>   In that case, why not just pass them with LDFLAGS?
>>
>
> Our SW is build in 2 parts: the Linux kernel, buildroot and other
> components are build via buildroot itself.
> Next to this we also have other applications that we build in with "a
> private build infrastructure" using the toolchain, include files etc
> of buildroot.
>
> By adding the extra linker parameters with the wrapper construction we
> can transparantly introduce a new toolchain (with the required
> parameters) in that environment without impact on our build
> infrastructure. The big advantage is that the complete toolchain
> selection (CPU,ABI etc) only needs to be done in buildroot.

  Yes, sounds like a valid use case.

  I also expect that the LDFLAGS are not passed reliably for all packages:
they are hardly ever used, and there are no autobuilder configurations with
any LDFLAGS, so errors in passing LDFLAGS can go unnoticed.

  I don't think we have any other architecture currently in buildroot that
requires something to be passed to ld. If this wrapper construction is
added to buildroot, it would be good if we would also have an internal
use case.

  Regards,
  Arnout
Ronny Meeus - Nov. 26, 2012, 8:45 p.m.
On Sun, Nov 25, 2012 at 11:25 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 25/11/12 21:56, Ronny Meeus wrote:
>>
>> On Sun, Nov 25, 2012 at 9:44 PM, Arnout Vandecappelle<arnout@mind.be>
>> wrote:
>
> [snip]
>
>>>   In that case, why not just pass them with LDFLAGS?
>>>
>>
>> Our SW is build in 2 parts: the Linux kernel, buildroot and other
>> components are build via buildroot itself.
>> Next to this we also have other applications that we build in with "a
>> private build infrastructure" using the toolchain, include files etc
>> of buildroot.
>>
>> By adding the extra linker parameters with the wrapper construction we
>> can transparantly introduce a new toolchain (with the required
>> parameters) in that environment without impact on our build
>> infrastructure. The big advantage is that the complete toolchain
>> selection (CPU,ABI etc) only needs to be done in buildroot.
>
>
>  Yes, sounds like a valid use case.
>
>  I also expect that the LDFLAGS are not passed reliably for all packages:
> they are hardly ever used, and there are no autobuilder configurations with
> any LDFLAGS, so errors in passing LDFLAGS can go unnoticed.
>
>  I don't think we have any other architecture currently in buildroot that
> requires something to be passed to ld. If this wrapper construction is
> added to buildroot, it would be good if we would also have an internal
> use case.
>

I do not really understand. On one hand you say it is a valid use-case
and on the other hand you say that an internal use-case is needed.
Shall I put some more effort in generalising the C code of the
wrapper?
Is there any chance that it gets accepted by the community?

---
Ronny

>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                               arnout at mind be
> Senior Embedded Software Architect                 +32-16-286540
> Essensium/Mind                                     http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Arnout Vandecappelle - Nov. 26, 2012, 10:33 p.m.
On 26/11/12 21:45, Ronny Meeus wrote:
>> >    I don't think we have any other architecture currently in buildroot that
>> >  requires something to be passed to ld. If this wrapper construction is
>> >  added to buildroot, it would be good if we would also have an internal
>> >  use case.
>> >
> I do not really understand. On one hand you say it is a valid use-case
> and on the other hand you say that an internal use-case is needed.

  If ld requires some architecture specific flag to work, then it's a valid
use case to add this in the wrapper. (However, I now realize that this
would only work for external toolchains...  For internal and crosstool-NG
toolchains, we don't have a wrapper and it's not needed because those
things are hard-coded at compile time. So how would you deal with that for
this ld case?)

  But it would improve acceptance if there were also a use case inside
buildroot. I tried to find a package that would also require the additional
LD flag, but it seems that most packages that do partial linking use
$(CC) -nostdlib.

> Shall I put some more effort in generalising the C code of the
> wrapper?
> Is there any chance that it gets accepted by the community?

  If you can show a .config that fails to build without it: certainly.
Otherwise, it depends on how hard you push it :-)

  Regards,
  Arnout
Ronny Meeus - Nov. 28, 2012, 7:38 p.m.
On Mon, Nov 26, 2012 at 11:33 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 26/11/12 21:45, Ronny Meeus wrote:
>>>
>>> >    I don't think we have any other architecture currently in buildroot
>>> > that
>>> >  requires something to be passed to ld. If this wrapper construction is
>>> >  added to buildroot, it would be good if we would also have an internal
>>> >  use case.
>>> >
>>
>> I do not really understand. On one hand you say it is a valid use-case
>> and on the other hand you say that an internal use-case is needed.
>
>
>  If ld requires some architecture specific flag to work, then it's a valid
> use case to add this in the wrapper. (However, I now realize that this
> would only work for external toolchains...  For internal and crosstool-NG
> toolchains, we don't have a wrapper and it's not needed because those
> things are hard-coded at compile time. So how would you deal with that for
> this ld case?)
>
>  But it would improve acceptance if there were also a use case inside
> buildroot. I tried to find a package that would also require the additional
> LD flag, but it seems that most packages that do partial linking use
> $(CC) -nostdlib.
>
>
>> Shall I put some more effort in generalising the C code of the
>> wrapper?
>> Is there any chance that it gets accepted by the community?
>
>
>  If you can show a .config that fails to build without it: certainly.
> Otherwise, it depends on how hard you push it :-)
>
>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                               arnout at mind be
> Senior Embedded Software Architect                 +32-16-286540
> Essensium/Mind                                     http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

I will update the private build-infrastructure, so there is no need to
change anything to buildroot.

Thanks for the support.
---
Ronny

Patch

diff --git a/toolchain/toolchain-external/ext-toolchain-wrapper.c
b/toolchain/toolchain-external/ext-toolchain-wrapper.c
--- a/toolchain/toolchain-external/ext-toolchain-wrapper.c
+++ b/toolchain/toolchain-external/ext-toolchain-wrapper.c
@@ -19,7 +19,7 @@ 

 static char path[PATH_MAX] = BR_CROSS_PATH;

-static char *predef_args[] = {
+static char *predef_args_cc[] = {
        path,
        "--sysroot", BR_SYSROOT,
 #ifdef BR_ARCH
@@ -48,6 +48,11 @@  static char *predef_args[] = {
 #endif
 };

+static char *predef_args_lnk[] = {
+       path,
+       "-melf32btsmipn32",
+};
+
 static const char *get_basename(const char *name)
 {
        const char *base;
@@ -64,16 +69,24 @@  static const char *get_basename(const ch
 int main(int argc, char **argv)
 {
        char **args, **cur;
+       char **predef_args = predef_args_cc;
+       int size_args = sizeof(predef_args_cc);

-       cur = args = malloc(sizeof(predef_args) + (sizeof(char *) * argc));
+       if (strcmp(argv[0]+strlen(argv[0])-3,"-ld") == 0) {
+               predef_args = predef_args_lnk;
+               size_args = sizeof(predef_args_lnk);
+       }
+
+       cur = args = malloc(size_args + (sizeof(char *) * argc));
        if (args == NULL) {
                perror(__FILE__ ": malloc");
                return 2;
        }

        /* start with predefined args */
-       memcpy(cur, predef_args, sizeof(predef_args));
-       cur += sizeof(predef_args) / sizeof(predef_args[0]);
+       memcpy(cur, predef_args, size_args);
+       cur += size_args / sizeof(predef_args[0]);

        /* append forward args */
        memcpy(cur, &argv[1], sizeof(char *) * (argc - 1));


diff --git a/toolchain/toolchain-external/ext-tool.mk
b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -440,7 +440,7 @@  endif
        for i in $(TOOLCHAIN_EXTERNAL_CROSS)*; do \
                base=$${i##*/}; \
                case "$$base" in \
-               *cc|*cc-*|*++|*++-*|*cpp) \
+               *cc|*cc-*|*++|*++-*|*cpp|*ld) \
                        ln -sf $(@F) $$base; \
                        ;; \
                *) \