[net-next,wrong] make BPFILTER_UMH depend on X86
diff mbox series

Message ID 20180528153222.2037158-1-arnd@arndb.de
State RFC, archived
Delegated to: David Miller
Headers show
Series
  • [net-next,wrong] make BPFILTER_UMH depend on X86
Related show

Commit Message

Arnd Bergmann May 28, 2018, 3:31 p.m. UTC
When build testing across architectures, I run into a build error on
all targets other than X86:

gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target

The problem is that 'hostprogs' get built with 'gcc' rather than
'$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.

To work around it, adding an X86 dependency gets randconfigs building
again on my box.

Clearly, this is not a good solution, since it should actually work fine
when building native kernels on other architectures but that is now
disabled, while cross building an x86 kernel on another host is still
broken after my patch.

What we probably want here is to try out if the compiler is able to build
executables for the target architecture and not build the helper otherwise,
at least when compile-testing. No idea how to do that though.

Link: http://www.kernel.org/pub/tools/crosstool/
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 net/bpfilter/Kconfig | 1 +
 1 file changed, 1 insertion(+)

Comments

Alexei Starovoitov May 30, 2018, 3:17 p.m. UTC | #1
On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
> When build testing across architectures, I run into a build error on
> all targets other than X86:
> 
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
> 
> The problem is that 'hostprogs' get built with 'gcc' rather than
> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
> 
> To work around it, adding an X86 dependency gets randconfigs building
> again on my box.
> 
> Clearly, this is not a good solution, since it should actually work fine
> when building native kernels on other architectures but that is now
> disabled, while cross building an x86 kernel on another host is still
> broken after my patch.
> 
> What we probably want here is to try out if the compiler is able to build
> executables for the target architecture and not build the helper otherwise,
> at least when compile-testing. No idea how to do that though.
> 
> Link: http://www.kernel.org/pub/tools/crosstool/
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: linux-kbuild@vger.kernel.org
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
>  net/bpfilter/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
> index 60725c5f79db..61cc4fcbb4d0 100644
> --- a/net/bpfilter/Kconfig
> +++ b/net/bpfilter/Kconfig
> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>  if BPFILTER
>  config BPFILTER_UMH
>  	tristate "bpfilter kernel module with user mode helper"
> +	depends on X86 # actually depends on native builds

depends on X86 will break it on arm.
I think the better short term fix would be to test that HOSTCC == CC
It doesn't have to be the same compiler. HOSTCC's arch == kernel ARCH
Not sure how to hack makefile to do that.
Long term we need to get rid of HOSTCC dependency.
Masahiro Yamada May 31, 2018, 1:42 a.m. UTC | #2
2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <alexei.starovoitov@gmail.com>:
> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
>> When build testing across architectures, I run into a build error on
>> all targets other than X86:
>>
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>
>> The problem is that 'hostprogs' get built with 'gcc' rather than
>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>
>> To work around it, adding an X86 dependency gets randconfigs building
>> again on my box.
>>
>> Clearly, this is not a good solution, since it should actually work fine
>> when building native kernels on other architectures but that is now
>> disabled, while cross building an x86 kernel on another host is still
>> broken after my patch.
>>
>> What we probably want here is to try out if the compiler is able to build
>> executables for the target architecture and not build the helper otherwise,
>> at least when compile-testing. No idea how to do that though.
>>
>> Link: http://www.kernel.org/pub/tools/crosstool/
>> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
>> Cc: linux-kbuild@vger.kernel.org
>> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
>> ---
>>  net/bpfilter/Kconfig | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
>> index 60725c5f79db..61cc4fcbb4d0 100644
>> --- a/net/bpfilter/Kconfig
>> +++ b/net/bpfilter/Kconfig
>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>>  if BPFILTER
>>  config BPFILTER_UMH
>>       tristate "bpfilter kernel module with user mode helper"
>> +     depends on X86 # actually depends on native builds
>
> depends on X86 will break it on arm.
> I think the better short term fix would be to test that HOSTCC == CC
> It doesn't have to be the same compiler. HOSTCC's arch == kernel ARCH
> Not sure how to hack makefile to do that.
> Long term we need to get rid of HOSTCC dependency.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hmm.
For cross-compiling, we set 'ARCH' via the environment variable or the
command line.

ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)


ARCH ?= $(SUBARCH)


Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
Arnd Bergmann June 1, 2018, 3:20 p.m. UTC | #3
On Thu, May 31, 2018 at 3:42 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <alexei.starovoitov@gmail.com>:
>> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:

> Hmm.
> For cross-compiling, we set 'ARCH' via the environment variable or the
> command line.
>
> ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)
>
>
> ARCH ?= $(SUBARCH)
>
>
> Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
>

SUBARCH is also used with a special meaning for arch/um where we build
with ARCH=um SUBARCH=x86, either on native (x86) or cross builds.


So doing that would still work in most but not all cases.

What is the reason for using HOSTCC rather than CC anyway? I think
the correct way to do this would be to check if CC is able to link binaries
and disallow the option if it's not.

Don't we already do something like that for tools/testing/selftest which
also needs to generate binaries with CC?

       Arnd
Alexei Starovoitov June 4, 2018, 11:51 p.m. UTC | #4
On Fri, Jun 01, 2018 at 05:20:12PM +0200, Arnd Bergmann wrote:
> On Thu, May 31, 2018 at 3:42 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
> > 2018-05-31 0:17 GMT+09:00 Alexei Starovoitov <alexei.starovoitov@gmail.com>:
> >> On Mon, May 28, 2018 at 05:31:01PM +0200, Arnd Bergmann wrote:
> 
> > Hmm.
> > For cross-compiling, we set 'ARCH' via the environment variable or the
> > command line.
> >
> > ARCH is not explicitly set, the top-level Makefile sets it to $(SUBARCH)
> >
> >
> > ARCH ?= $(SUBARCH)
> >
> >
> > Maybe, we can assume the native build if $(ARCH) and $(SUBARCH) are the same?
> >
> 
> SUBARCH is also used with a special meaning for arch/um where we build
> with ARCH=um SUBARCH=x86, either on native (x86) or cross builds.
> 
> 
> So doing that would still work in most but not all cases.
> 
> What is the reason for using HOSTCC rather than CC anyway? I think
> the correct way to do this would be to check if CC is able to link binaries
> and disallow the option if it's not.

that's a great idea. Let's do that.

> Don't we already do something like that for tools/testing/selftest which
> also needs to generate binaries with CC?

I couldn't find such makefile magic. Can you please help me with this?
Geert Uytterhoeven June 8, 2018, 8:57 a.m. UTC | #5
Hi Arnd, Alexei,

On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> When build testing across architectures, I run into a build error on
> all targets other than X86:
>
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>
> The problem is that 'hostprogs' get built with 'gcc' rather than
> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>
> To work around it, adding an X86 dependency gets randconfigs building
> again on my box.
>
> Clearly, this is not a good solution, since it should actually work fine
> when building native kernels on other architectures but that is now
> disabled, while cross building an x86 kernel on another host is still
> broken after my patch.
>
> What we probably want here is to try out if the compiler is able to build
> executables for the target architecture and not build the helper otherwise,
> at least when compile-testing. No idea how to do that though.

So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
from HOSTCC"), but it is not sufficient:

      GEN net/bpfilter/bpfilter_umh.o
    Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
     Copies a binary file, possibly transforming it in the process
     The options are:
    [...]

net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1

> --- a/net/bpfilter/Kconfig
> +++ b/net/bpfilter/Kconfig
> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>  if BPFILTER
>  config BPFILTER_UMH
>         tristate "bpfilter kernel module with user mode helper"
> +       depends on X86 # actually depends on native builds

No, (currently) it does depend on X86, due to its use of:

    $(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)

with CONFIG_OUTPUT_FORMAT being defined on X86 only...

>         default m
>         help
>           This builds bpfilter kernel module with embedded user mode helper

Gr{oetje,eeting}s,

                        Geert
Daniel Borkmann June 8, 2018, 9:33 a.m. UTC | #6
Hi Geert,

On 06/08/2018 10:57 AM, Geert Uytterhoeven wrote:
> On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>> When build testing across architectures, I run into a build error on
>> all targets other than X86:
>>
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>
>> The problem is that 'hostprogs' get built with 'gcc' rather than
>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>
>> To work around it, adding an X86 dependency gets randconfigs building
>> again on my box.
>>
>> Clearly, this is not a good solution, since it should actually work fine
>> when building native kernels on other architectures but that is now
>> disabled, while cross building an x86 kernel on another host is still
>> broken after my patch.
>>
>> What we probably want here is to try out if the compiler is able to build
>> executables for the target architecture and not build the helper otherwise,
>> at least when compile-testing. No idea how to do that though.
> 
> So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
> from HOSTCC"), but it is not sufficient:
> 
>       GEN net/bpfilter/bpfilter_umh.o
>     Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
>      Copies a binary file, possibly transforming it in the process
>      The options are:
>     [...]
> 
> net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
> make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1
> 
>> --- a/net/bpfilter/Kconfig
>> +++ b/net/bpfilter/Kconfig
>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>>  if BPFILTER
>>  config BPFILTER_UMH
>>         tristate "bpfilter kernel module with user mode helper"
>> +       depends on X86 # actually depends on native builds
> 
> No, (currently) it does depend on X86, due to its use of:
> 
>     $(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)
> 
> with CONFIG_OUTPUT_FORMAT being defined on X86 only...

That hard dependency should have been fixed with the following patch in -net tree:

https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=8d97ca6b6755bf7ef57d323642ca9ee80d689782

Thanks,
Daniel
Geert Uytterhoeven June 8, 2018, 9:40 a.m. UTC | #7
Hi Daniel,

On Fri, Jun 8, 2018 at 11:33 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/08/2018 10:57 AM, Geert Uytterhoeven wrote:
>> On Mon, May 28, 2018 at 5:31 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>>> When build testing across architectures, I run into a build error on
>>> all targets other than X86:
>>>
>>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objdump: net/bpfilter/bpfilter_umh: File format not recognized
>>> gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy:net/bpfilter/bpfilter_umh.o: Invalid bfd target
>>>
>>> The problem is that 'hostprogs' get built with 'gcc' rather than
>>> '$(CROSS_COMPILE)gcc', and my default gcc (as most people's) targets x86.
>>>
>>> To work around it, adding an X86 dependency gets randconfigs building
>>> again on my box.
>>>
>>> Clearly, this is not a good solution, since it should actually work fine
>>> when building native kernels on other architectures but that is now
>>> disabled, while cross building an x86 kernel on another host is still
>>> broken after my patch.
>>>
>>> What we probably want here is to try out if the compiler is able to build
>>> executables for the target architecture and not build the helper otherwise,
>>> at least when compile-testing. No idea how to do that though.
>>
>> So that was done in commit 819dd92b9c0bc7bc ("bpfilter: switch to CC
>> from HOSTCC"), but it is not sufficient:
>>
>>       GEN net/bpfilter/bpfilter_umh.o
>>     Usage: m68k-linux-gnu-objcopy [option(s)] in-file [out-file]
>>      Copies a binary file, possibly transforming it in the process
>>      The options are:
>>     [...]
>>
>> net/bpfilter/Makefile:29: recipe for target 'net/bpfilter/bpfilter_umh.o' failed
>> make[5]: *** [net/bpfilter/bpfilter_umh.o] Error 1
>>
>>> --- a/net/bpfilter/Kconfig
>>> +++ b/net/bpfilter/Kconfig
>>> @@ -9,6 +9,7 @@ menuconfig BPFILTER
>>>  if BPFILTER
>>>  config BPFILTER_UMH
>>>         tristate "bpfilter kernel module with user mode helper"
>>> +       depends on X86 # actually depends on native builds
>>
>> No, (currently) it does depend on X86, due to its use of:
>>
>>     $(OBJCOPY) -I binary -O $(CONFIG_OUTPUT_FORMAT)
>>
>> with CONFIG_OUTPUT_FORMAT being defined on X86 only...
>
> That hard dependency should have been fixed with the following patch in -net tree:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=8d97ca6b6755bf7ef57d323642ca9ee80d689782

Thanks, confirmed.

Gr{oetje,eeting}s,

                        Geert

Patch
diff mbox series

diff --git a/net/bpfilter/Kconfig b/net/bpfilter/Kconfig
index 60725c5f79db..61cc4fcbb4d0 100644
--- a/net/bpfilter/Kconfig
+++ b/net/bpfilter/Kconfig
@@ -9,6 +9,7 @@  menuconfig BPFILTER
 if BPFILTER
 config BPFILTER_UMH
 	tristate "bpfilter kernel module with user mode helper"
+	depends on X86 # actually depends on native builds
 	default m
 	help
 	  This builds bpfilter kernel module with embedded user mode helper