[v5] dpdk: new package
diff mbox

Message ID 1460826490-15614-1-git-send-email-viktorin@rehivetech.com
State Changes Requested
Headers show

Commit Message

Jan Viktorin April 16, 2016, 5:08 p.m. UTC
This patch introduces support of the DPDK library (www.dpdk.org) into
Buildroot. DPDK is a library for high-speed packet sending/receiving
while bypassing the Linux Kernel. It allows to reach a high throughput
for 10-100 Gbps networks on the x86 platform.

The package compiles and installs DPDK libraries on the target and
staging and allows to compile other applications depending on the DPDK
library. It can also install some basic tools the DPDK provides
(testpmd, python scripts, test suite).

The patch assumes DPDK 16.04. This version contains support for the ARM
architecture. The ARM ports can be tested by

 qemu_aarch64_virt_defconfig
 qemu_arm_vexpress_defconfig

The included hash was calculated locally by downloading the tar.gz archives by
hand.

There are unfortunately some pitfalls:

* it may require to enable PCI, MSIX, UIO in the Linux Kernel
  (some defconfigs does not include as default and it is platform
  dependent as ARMv7 almost does not use PCI)

* when building PCAP PMD driver, the libpcap is required (partially
  fixed as suggested by Arnout)

* some tools the DPDK provides depend on Python(2) so the user has
  to enable it to install those

Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
---
v2: (mostly suggestions by Arnout)
* simplified Config.in - avoid version, source and config selection
* improved dependency on libpcap
* user python scripts are included if python package is enabled
* installing of tests includes autotest*.py scripts (we do not care
  about the python here, assuming the user will install it manually
  when testing)
* minor coding style fixes
* depends on python-pexpect package
* using version 2.2.0-rc3
v3
* bump to version 16.04-rc1
* utilizing the new install syntax of DPDK
* fixed line wrapping
* testpmd is now always installed (no option for this)
* fixed python dependencies (T. Petazzoni)
* supporting i686+ architectures (T. Petazzoni)
* dropped PPC support for now
* utilizing KCONFIG_ENABLE_OPT macro
* support for build & install of examples (ugly code)
* build with MAKE instead of MAKE1 seems OK now
* dropped COMBINE_LIBS (no more in DPDK)
* added HAS_SYNC_x
* few random fixes
v4 (suggestions by T. Petazzoni)
* bump to version 16.04-rc2
* fixed indentation in Config.in
* fixed depends in Config.in
* handling kernel modules by BR2_LINUX_NEEDS_MODULES
* simplified installation of examples
v5
* bump to version 16.04
* dropped (forgotten) LINUX_NEEDS_MODULES=y from mk
* reordered Config.in (T. Petazzoni)
* added forgotten BR2_x86_64 (T. Petazzoni)
* dropped support for examples (T. Petazzoni)
* fix install of kernel modules (T. Petazzoni)
* patch ARM ABI enforcing (T. Petazzoni)
* minor install fixes (T. Petazzoni)
---
 package/Config.in                                  |  1 +
 ...01-mk-do-not-enforce-any-specific-ARM-ABI.patch | 31 +++++++++++
 package/dpdk/Config.in                             | 60 ++++++++++++++++++++++
 package/dpdk/dpdk.hash                             |  2 +
 package/dpdk/dpdk.mk                               | 60 ++++++++++++++++++++++
 5 files changed, 154 insertions(+)
 create mode 100644 package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
 create mode 100644 package/dpdk/Config.in
 create mode 100644 package/dpdk/dpdk.hash
 create mode 100644 package/dpdk/dpdk.mk

Comments

Thomas Petazzoni April 17, 2016, 2:38 p.m. UTC | #1
Hello,

On Sat, 16 Apr 2016 19:08:10 +0200, Jan Viktorin wrote:
> This patch introduces support of the DPDK library (www.dpdk.org) into
> Buildroot. DPDK is a library for high-speed packet sending/receiving
> while bypassing the Linux Kernel. It allows to reach a high throughput
> for 10-100 Gbps networks on the x86 platform.
> 
> The package compiles and installs DPDK libraries on the target and
> staging and allows to compile other applications depending on the DPDK
> library. It can also install some basic tools the DPDK provides
> (testpmd, python scripts, test suite).
> 
> The patch assumes DPDK 16.04. This version contains support for the ARM
> architecture. The ARM ports can be tested by
> 
>  qemu_aarch64_virt_defconfig
>  qemu_arm_vexpress_defconfig
> 
> The included hash was calculated locally by downloading the tar.gz archives by
> hand.
> 
> There are unfortunately some pitfalls:
> 
> * it may require to enable PCI, MSIX, UIO in the Linux Kernel
>   (some defconfigs does not include as default and it is platform
>   dependent as ARMv7 almost does not use PCI)
> 
> * when building PCAP PMD driver, the libpcap is required (partially
>   fixed as suggested by Arnout)
> 
> * some tools the DPDK provides depend on Python(2) so the user has
>   to enable it to install those
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>

Thanks for this new iteration. Unfortunately, it still doesn't install
any kernel module to my system. At install time, your addition of the
install-kmod target simply does:

make[3]: Nothing to be done for 'install-kmod'.

Moreover, are you sure having a Linux kernel already built is always
necessary? My understanding is that while DPDK has its own kernel
modules, it can also rely on the kernel generic uio_pci_generic module.
In this case, DPDK only needs to build userspace components, and no
kernel component, which would make the dependency on Linux optional.

From the DPDK documentation: "To run any DPDK application, a suitable
uio module can be loaded into the running kernel. In many cases, the
standard uio_pci_generic module included in the Linux kernel can
provide the uio capability.".

Best regards,

Thomas
Jan Viktorin April 17, 2016, 3:56 p.m. UTC | #2
Hello Thomas,

what configuration do you use?

In general, I cannot say‎ we are always fine with the in-kernel modules. There are configurations of DPDK that build certain drivers. Of course, DPDK community would like to have no out-of-tree drivers. Some devices require a certain driver.

The x86 configuration builds drivers. The armv7 one doesn't (there are no useful drivers for this purpose).

A user may serve its own configuration of DPDK where a kernel driver is selected and thus, the user would expect it to be built and installed.

It seems that in the future the in-kernel vfio is the way to‎ go.

Jan Viktorin
RehiveTech
Sent from a mobile device
  Původní zpráva  
Od: Thomas Petazzoni
Odesláno: neděle, 17. dubna 2016 16:38
Komu: Jan Viktorin
Kopie: buildroot@buildroot.org
Předmět: Re: [Buildroot] [PATCH v5] dpdk: new package

Hello,

On Sat, 16 Apr 2016 19:08:10 +0200, Jan Viktorin wrote:
> This patch introduces support of the DPDK library (www.dpdk.org) into
> Buildroot. DPDK is a library for high-speed packet sending/receiving
> while bypassing the Linux Kernel. It allows to reach a high throughput
> for 10-100 Gbps networks on the x86 platform.
> 
> The package compiles and installs DPDK libraries on the target and
> staging and allows to compile other applications depending on the DPDK
> library. It can also install some basic tools the DPDK provides
> (testpmd, python scripts, test suite).
> 
> The patch assumes DPDK 16.04. This version contains support for the ARM
> architecture. The ARM ports can be tested by
> 
> qemu_aarch64_virt_defconfig
> qemu_arm_vexpress_defconfig
> 
> The included hash was calculated locally by downloading the tar.gz archives by
> hand.
> 
> There are unfortunately some pitfalls:
> 
> * it may require to enable PCI, MSIX, UIO in the Linux Kernel
> (some defconfigs does not include as default and it is platform
> dependent as ARMv7 almost does not use PCI)
> 
> * when building PCAP PMD driver, the libpcap is required (partially
> fixed as suggested by Arnout)
> 
> * some tools the DPDK provides depend on Python(2) so the user has
> to enable it to install those
> 
> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>

Thanks for this new iteration. Unfortunately, it still doesn't install
any kernel module to my system. At install time, your addition of the
install-kmod target simply does:

make[3]: Nothing to be done for 'install-kmod'.

Moreover, are you sure having a Linux kernel already built is always
necessary? My understanding is that while DPDK has its own kernel
modules, it can also rely on the kernel generic uio_pci_generic module.
In this case, DPDK only needs to build userspace components, and no
kernel component, which would make the dependency on Linux optional.

From the DPDK documentation: "To run any DPDK application, a suitable
uio module can be loaded into the running kernel. In many cases, the
standard uio_pci_generic module included in the Linux kernel can
provide the uio capability.".

Best regards,

Thomas
Thomas Petazzoni April 17, 2016, 7:35 p.m. UTC | #3
Hello,

On Sun, 17 Apr 2016 17:56:07 +0200, Jan Viktorin wrote:

> what configuration do you use?

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2016.02-3-g762b7c9.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.5"
BR2_LINUX_KERNEL_DEFCONFIG="mvebu_v7"
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_DPDK=y
# BR2_TARGET_ROOTFS_TAR is not set

Before starting the build, I do "make linux-menuconfig", and enable
CONFIG_UIO in the kernel configuration.

> In general, I cannot say‎ we are always fine with the in-kernel
> modules. There are configurations of DPDK that build certain drivers.
> Of course, DPDK community would like to have no out-of-tree drivers.
> Some devices require a certain driver.
> 
> The x86 configuration builds drivers. The armv7 one doesn't (there
> are no useful drivers for this purpose).

Then IMO the dependency on the Linux kernel should be optional.

> A user may serve its own configuration of DPDK where a kernel driver
> is selected and thus, the user would expect it to be built and
> installed.

Right. But if the default configuration doesn't require the kernel to
be built, we probably shouldn't require it as well.

Thomas
Jan Viktorin April 17, 2016, 8:56 p.m. UTC | #4
Well, your config is for ARMv7 so no kmods are built because all current drivers require PCI which is hardly to see on this platform (I know just about Armada).

Ok. I don't know what would be the use case for dropping dependency on the kernel. Building just the rootfs without the kernel?

That means to check whether the kernel is enabled. And if it is then to enable building of the DPDK kernel modules. Right?

This can be done but I am afraid I have to disable certain modules by name from the current configuration. This is not very generic (consider adding a new out-of-tree kernel module or removing an obsolete one from the DPDK, custom DPDK tree with third party modules) but there are only few of them so it might be a working solution. I will check if there is a more generic way but I don't think so.
‎
Jan Viktorin
RehiveTech‎
Sent from a mobile device
  Původní zpráva  
Od: Thomas Petazzoni
Odesláno: neděle, 17. dubna 2016 21:35
Komu: Jan Viktorin
Kopie: buildroot@buildroot.org
Předmět: Re: [Buildroot] [PATCH v5] dpdk: new package

Hello,

On Sun, 17 Apr 2016 17:56:07 +0200, Jan Viktorin wrote:

> what configuration do you use?

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2016.02-3-g762b7c9.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.5"
BR2_LINUX_KERNEL_DEFCONFIG="mvebu_v7"
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_DPDK=y
# BR2_TARGET_ROOTFS_TAR is not set

Before starting the build, I do "make linux-menuconfig", and enable
CONFIG_UIO in the kernel configuration.

> In general, I cannot say‎ we are always fine with the in-kernel
> modules. There are configurations of DPDK that build certain drivers.
> Of course, DPDK community would like to have no out-of-tree drivers.
> Some devices require a certain driver.
> 
> The x86 configuration builds drivers. The armv7 one doesn't (there
> are no useful drivers for this purpose).

Then IMO the dependency on the Linux kernel should be optional.

> A user may serve its own configuration of DPDK where a kernel driver
> is selected and thus, the user would expect it to be built and
> installed.

Right. But if the default configuration doesn't require the kernel to
be built, we probably shouldn't require it as well.

Thomas
Thomas Petazzoni April 17, 2016, 9:06 p.m. UTC | #5
Hello,

On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:

> Well, your config is for ARMv7 so no kmods are built because all
> current drivers require PCI which is hardly to see on this platform
> (I know just about Armada).

Indeed mvebu_v7 is for Marvell Armada. Is DPDK only for PCI NICs? Can't
it be used for the internal NICs of certain ARM SoCs ?

> Ok. I don't know what would be the use case for dropping dependency
> on the kernel. Building just the rootfs without the kernel?

For example, yes. Another benefit of not having the mandatory kernel
dependency is that the autobuilders would be able to test the DPDK
package.

> That means to check whether the kernel is enabled. And if it is then
> to enable building of the DPDK kernel modules. Right?

Correct.

> This can be done but I am afraid I have to disable certain modules by
> name from the current configuration. This is not very generic
> (consider adding a new out-of-tree kernel module or removing an
> obsolete one from the DPDK, custom DPDK tree with third party
> modules) but there are only few of them so it might be a working
> solution. I will check if there is a more generic way but I don't
> think so.

I'm not sure what you mean here. If the default configuration on a
given architecture builds kernel modules, then on this architecture,
DPDK should depend on the kernel to be built. If the default
configuration on another architecture does not build kernel modules,
then there is no need to depend on the kernel.

That being said, if only ARM does not build any kernel module, maybe we
can simply do as you do right now, and depend on the kernel
unconditionally, to keep things simple.

Thanks!

Thomas
Jan Viktorin April 18, 2016, 8:23 a.m. UTC | #6
On Sun, 17 Apr 2016 23:06:07 +0200
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Hello,
> 
> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> 
> > Well, your config is for ARMv7 so no kmods are built because all
> > current drivers require PCI which is hardly to see on this platform
> > (I know just about Armada).  
> 
> Indeed mvebu_v7 is for Marvell Armada. Is DPDK only for PCI NICs? Can't
> it be used for the internal NICs of certain ARM SoCs ?

Yes. DPDK up to 16.04 supports only PCI devices. We are going to change
this for 16.07 (the next release).

> 
> > Ok. I don't know what would be the use case for dropping dependency
> > on the kernel. Building just the rootfs without the kernel?  
> 
> For example, yes. Another benefit of not having the mandatory kernel
> dependency is that the autobuilders would be able to test the DPDK
> package.

I have no clue about this. Does kernel dependency prevent some
auto-testing?

> 
> > That means to check whether the kernel is enabled. And if it is then
> > to enable building of the DPDK kernel modules. Right?  
> 
> Correct.

I would implement this in Buildroot by a list of configuration keys
naming certain drivers. If the kernel is disabled, I'd disable all of
them during configure phase. Otherwise, I'd leave them untouched (to
have the current configuration as is). So, I'd put in dpdk.mk
something like:

ifeq ($(BR2_LINUX_KERNEL),y)
DPDK_DEPENDENCIES += linux
else
DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0

define DPDK_DISABLE_KMODS
$(foreach m,$(DPDK_KMODS),\
	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
endef

DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS
endif

Probably, I can improve it by testing whether a module is present
in the DPDK config and then setting the dependency on linux.

Anyway, it seems to be quite complex...

> 
> > This can be done but I am afraid I have to disable certain modules by
> > name from the current configuration. This is not very generic
> > (consider adding a new out-of-tree kernel module or removing an
> > obsolete one from the DPDK, custom DPDK tree with third party
> > modules) but there are only few of them so it might be a working
> > solution. I will check if there is a more generic way but I don't
> > think so.  
> 
> I'm not sure what you mean here. If the default configuration on a
> given architecture builds kernel modules, then on this architecture,
> DPDK should depend on the kernel to be built. If the default
> configuration on another architecture does not build kernel modules,
> then there is no need to depend on the kernel.
> 
> That being said, if only ARM does not build any kernel module, maybe we
> can simply do as you do right now, and depend on the kernel
> unconditionally, to keep things simple.

Yes, this would work for the existing defconfigs. What about a custom
one? E.g. I want some DPDK kernel modules for the Armada with a
PCI-E NIC. So, I need a custom DPDK configuration at the moment.
Fortunately, this is a rare case.

There is also a Xen module in DPDK that is not built by default so you
have to create a custom configuration (or alter an existing one) to
enable it.

I understand that you'd like to keep it as simple as possible. I'd like
to keep it simple too but also providing enough flexibility for easy
customizations. That's why I don't consider the ARMv7 configuration to
be "without kernel modules" as it is possible to enable some if needed.

Regards
Jan

> 
> Thanks!
> 
> Thomas
Arnout Vandecappelle April 18, 2016, 10:50 p.m. UTC | #7
On 04/18/16 10:23, Jan Viktorin wrote:
> On Sun, 17 Apr 2016 23:06:07 +0200
> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
>
>> Hello,
>>
>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
>>
[snip]
>>> Ok. I don't know what would be the use case for dropping dependency
>>> on the kernel. Building just the rootfs without the kernel?
>>
>> For example, yes. Another benefit of not having the mandatory kernel
>> dependency is that the autobuilders would be able to test the DPDK
>> package.
>
> I have no clue about this. Does kernel dependency prevent some
> auto-testing?

  The base configs used in the autobuilders don't include a kernel (or 
bootloader or rootfs). Therefore, any package depending on the kernel is never 
built.

  We _could_ actually add a couple of base configs that do include a kernel. But 
someone has to do it :-)

>
>>
>>> That means to check whether the kernel is enabled. And if it is then
>>> to enable building of the DPDK kernel modules. Right?
>>
>> Correct.
>
> I would implement this in Buildroot by a list of configuration keys
> naming certain drivers. If the kernel is disabled, I'd disable all of
> them during configure phase. Otherwise, I'd leave them untouched (to
> have the current configuration as is). So, I'd put in dpdk.mk
> something like:
>
> ifeq ($(BR2_LINUX_KERNEL),y)
> DPDK_DEPENDENCIES += linux
> else
> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
>
> define DPDK_DISABLE_KMODS
> $(foreach m,$(DPDK_KMODS),\

  Small nit: indent with tab here.

> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> endef
>
> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS

  Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It 
gives you simple primitives for disabling and enabling options. Consult the manual.

> endif
>
> Probably, I can improve it by testing whether a module is present
> in the DPDK config and then setting the dependency on linux.

  That's not possible: DEPENDENCIES must be declared when the makefiles are 
parsed, but the tarball containing .config is only extracted afterwards.


  Regards,
  Arnout

>
> Anyway, it seems to be quite complex...
>
>>
>>> This can be done but I am afraid I have to disable certain modules by
>>> name from the current configuration. This is not very generic
>>> (consider adding a new out-of-tree kernel module or removing an
>>> obsolete one from the DPDK, custom DPDK tree with third party
>>> modules) but there are only few of them so it might be a working
>>> solution. I will check if there is a more generic way but I don't
>>> think so.
>>
>> I'm not sure what you mean here. If the default configuration on a
>> given architecture builds kernel modules, then on this architecture,
>> DPDK should depend on the kernel to be built. If the default
>> configuration on another architecture does not build kernel modules,
>> then there is no need to depend on the kernel.
>>
>> That being said, if only ARM does not build any kernel module, maybe we
>> can simply do as you do right now, and depend on the kernel
>> unconditionally, to keep things simple.
>
> Yes, this would work for the existing defconfigs. What about a custom
> one? E.g. I want some DPDK kernel modules for the Armada with a
> PCI-E NIC. So, I need a custom DPDK configuration at the moment.
> Fortunately, this is a rare case.
>
> There is also a Xen module in DPDK that is not built by default so you
> have to create a custom configuration (or alter an existing one) to
> enable it.
>
> I understand that you'd like to keep it as simple as possible. I'd like
> to keep it simple too but also providing enough flexibility for easy
> customizations. That's why I don't consider the ARMv7 configuration to
> be "without kernel modules" as it is possible to enable some if needed.
Jan Viktorin April 19, 2016, 12:27 p.m. UTC | #8
Hello Arnout,

thanks for your comments...

On Tue, 19 Apr 2016 00:50:27 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 04/18/16 10:23, Jan Viktorin wrote:
> > On Sun, 17 Apr 2016 23:06:07 +0200
> > Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> >  
> >> Hello,
> >>
> >> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> >>  
> [snip]
> >>> Ok. I don't know what would be the use case for dropping dependency
> >>> on the kernel. Building just the rootfs without the kernel?  
> >>
> >> For example, yes. Another benefit of not having the mandatory kernel
> >> dependency is that the autobuilders would be able to test the DPDK
> >> package.  
> >
> > I have no clue about this. Does kernel dependency prevent some
> > auto-testing?  
> 
>   The base configs used in the autobuilders don't include a kernel (or 
> bootloader or rootfs). Therefore, any package depending on the kernel is never 
> built.
> 
>   We _could_ actually add a couple of base configs that do include a kernel. But 
> someone has to do it :-)

I have no idea how to access the autobuilder nor what kind of complexity
is connected with this...

> 
> >  
> >>  
> >>> That means to check whether the kernel is enabled. And if it is then
> >>> to enable building of the DPDK kernel modules. Right?  
> >>
> >> Correct.  
> >
> > I would implement this in Buildroot by a list of configuration keys
> > naming certain drivers. If the kernel is disabled, I'd disable all of
> > them during configure phase. Otherwise, I'd leave them untouched (to
> > have the current configuration as is). So, I'd put in dpdk.mk
> > something like:
> >
> > ifeq ($(BR2_LINUX_KERNEL),y)
> > DPDK_DEPENDENCIES += linux
> > else
> > DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
> >
> > define DPDK_DISABLE_KMODS
> > $(foreach m,$(DPDK_KMODS),\  
> 
>   Small nit: indent with tab here.

OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.

> 
> > 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> > endef
> >
> > DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS  
> 
>   Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It 
> gives you simple primitives for disabling and enabling options. Consult the manual.

Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.

> 
> > endif
> >
> > Probably, I can improve it by testing whether a module is present
> > in the DPDK config and then setting the dependency on linux.  
> 
>   That's not possible: DEPENDENCIES must be declared when the makefiles are 
> parsed, but the tarball containing .config is only extracted afterwards.

OK, it was just an idea.

Regards
Jan

> 
> 
>   Regards,
>   Arnout
> 
> >
> > Anyway, it seems to be quite complex...
> >  
> >>  
> >>> This can be done but I am afraid I have to disable certain modules by
> >>> name from the current configuration. This is not very generic
> >>> (consider adding a new out-of-tree kernel module or removing an
> >>> obsolete one from the DPDK, custom DPDK tree with third party
> >>> modules) but there are only few of them so it might be a working
> >>> solution. I will check if there is a more generic way but I don't
> >>> think so.  
> >>
> >> I'm not sure what you mean here. If the default configuration on a
> >> given architecture builds kernel modules, then on this architecture,
> >> DPDK should depend on the kernel to be built. If the default
> >> configuration on another architecture does not build kernel modules,
> >> then there is no need to depend on the kernel.
> >>
> >> That being said, if only ARM does not build any kernel module, maybe we
> >> can simply do as you do right now, and depend on the kernel
> >> unconditionally, to keep things simple.  
> >
> > Yes, this would work for the existing defconfigs. What about a custom
> > one? E.g. I want some DPDK kernel modules for the Armada with a
> > PCI-E NIC. So, I need a custom DPDK configuration at the moment.
> > Fortunately, this is a rare case.
> >
> > There is also a Xen module in DPDK that is not built by default so you
> > have to create a custom configuration (or alter an existing one) to
> > enable it.
> >
> > I understand that you'd like to keep it as simple as possible. I'd like
> > to keep it simple too but also providing enough flexibility for easy
> > customizations. That's why I don't consider the ARMv7 configuration to
> > be "without kernel modules" as it is possible to enable some if needed.  
> 
> 
>
Arnout Vandecappelle April 19, 2016, 7:30 p.m. UTC | #9
On 04/19/16 14:27, Jan Viktorin wrote:
> Hello Arnout,
>
> thanks for your comments...
>
> On Tue, 19 Apr 2016 00:50:27 +0200
> Arnout Vandecappelle <arnout@mind.be> wrote:
>
>> On 04/18/16 10:23, Jan Viktorin wrote:
>>> On Sun, 17 Apr 2016 23:06:07 +0200
>>> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
>>>>
>> [snip]
>>>>> Ok. I don't know what would be the use case for dropping dependency
>>>>> on the kernel. Building just the rootfs without the kernel?
>>>>
>>>> For example, yes. Another benefit of not having the mandatory kernel
>>>> dependency is that the autobuilders would be able to test the DPDK
>>>> package.
>>>
>>> I have no clue about this. Does kernel dependency prevent some
>>> auto-testing?
>>
>>    The base configs used in the autobuilders don't include a kernel (or
>> bootloader or rootfs). Therefore, any package depending on the kernel is never
>> built.
>>
>>    We _could_ actually add a couple of base configs that do include a kernel. But
>> someone has to do it :-)
>
> I have no idea how to access the autobuilder nor what kind of complexity
> is connected with this...

  git://git.buildroot.net/buildroot-test/


>>> I would implement this in Buildroot by a list of configuration keys
>>> naming certain drivers. If the kernel is disabled, I'd disable all of
>>> them during configure phase. Otherwise, I'd leave them untouched (to
>>> have the current configuration as is). So, I'd put in dpdk.mk
>>> something like:
>>>
>>> ifeq ($(BR2_LINUX_KERNEL),y)
>>> DPDK_DEPENDENCIES += linux
>>> else
>>> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
>>>
>>> define DPDK_DISABLE_KMODS
>>> $(foreach m,$(DPDK_KMODS),\
>>
>>    Small nit: indent with tab here.
>
> OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.

  Yes it can.

  If you don't disable these options, what will the dpdk build system do? Error 
out? Or try to use the build system's kernel? Ideally there would be a way to 
globally tell it not to build modules. Manually maintaining a list of config 
options to disable is a bit cumbersome.


>
>>
>>> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
>>> endef
>>>
>>> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS
>>
>>    Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It
>> gives you simple primitives for disabling and enabling options. Consult the manual.
>
> Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.

  So you have to manually create a config file with lines like
# CONFIG_FOO is not set
? How uncomfortable...

>
>>
>>> endif
>>>
>>> Probably, I can improve it by testing whether a module is present
>>> in the DPDK config and then setting the dependency on linux.
>>
>>    That's not possible: DEPENDENCIES must be declared when the makefiles are
>> parsed, but the tarball containing .config is only extracted afterwards.
>
> OK, it was just an idea.

  We can also still revert to depending on linux unconditionally like you have 
now. How likely is it that dpdk will be used in practice with no kernel modules? 
Or is this going to be more likely in the future, with heavier use of UIO and 
device tree?


  Regards,
  Arnout
Thomas Petazzoni April 19, 2016, 8:47 p.m. UTC | #10
Hello,

On Tue, 19 Apr 2016 21:30:10 +0200, Arnout Vandecappelle wrote:

>   We can also still revert to depending on linux unconditionally like you have 
> now. How likely is it that dpdk will be used in practice with no kernel modules? 
> Or is this going to be more likely in the future, with heavier use of UIO and 
> device tree?

My proposal would indeed to revert back to depend on Linux
unconditionally, and see what happens later, like if we have users who
use DPDK and complain that it depends on the Linux kernel package.

Thomas
Jan Viktorin April 19, 2016, 9:41 p.m. UTC | #11
On Tue, 19 Apr 2016 21:30:10 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 04/19/16 14:27, Jan Viktorin wrote:
> > Hello Arnout,
> >
> > thanks for your comments...
> >
> > On Tue, 19 Apr 2016 00:50:27 +0200
> > Arnout Vandecappelle <arnout@mind.be> wrote:
> >  
> >> On 04/18/16 10:23, Jan Viktorin wrote:  
> >>> On Sun, 17 Apr 2016 23:06:07 +0200
> >>> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> >>>  
> >>>> Hello,
> >>>>
> >>>> On Sun, 17 Apr 2016 22:56:07 +0200, Jan Viktorin wrote:
> >>>>  
> >> [snip]  
> >>>>> Ok. I don't know what would be the use case for dropping dependency
> >>>>> on the kernel. Building just the rootfs without the kernel?  
> >>>>
> >>>> For example, yes. Another benefit of not having the mandatory kernel
> >>>> dependency is that the autobuilders would be able to test the DPDK
> >>>> package.  
> >>>
> >>> I have no clue about this. Does kernel dependency prevent some
> >>> auto-testing?  
> >>
> >>    The base configs used in the autobuilders don't include a kernel (or
> >> bootloader or rootfs). Therefore, any package depending on the kernel is never
> >> built.
> >>
> >>    We _could_ actually add a couple of base configs that do include a kernel. But
> >> someone has to do it :-)  
> >
> > I have no idea how to access the autobuilder nor what kind of complexity
> > is connected with this...  
> 
>   git://git.buildroot.net/buildroot-test/
> 
> 
> >>> I would implement this in Buildroot by a list of configuration keys
> >>> naming certain drivers. If the kernel is disabled, I'd disable all of
> >>> them during configure phase. Otherwise, I'd leave them untouched (to
> >>> have the current configuration as is). So, I'd put in dpdk.mk
> >>> something like:
> >>>
> >>> ifeq ($(BR2_LINUX_KERNEL),y)
> >>> DPDK_DEPENDENCIES += linux
> >>> else
> >>> DPDK_KMODS = EAL_IGB_UIO KNI_KMOD LIBRTE_XEN_DOM0
> >>>
> >>> define DPDK_DISABLE_KMODS
> >>> $(foreach m,$(DPDK_KMODS),\  
> >>
> >>    Small nit: indent with tab here.  
> >
> > OK, it was just an ad hoc example. I'd rather know whether I can implement it this way.  
> 
>   Yes it can.
> 
>   If you don't disable these options, what will the dpdk build system do? Error 
> out? Or try to use the build system's kernel? Ideally there would be a way to 

My experience is that if I don't pass the location of the kernel
sources it finds out some local one installed on the host system
or it errors out. Only the ARMv7 config builds as it disables all the
modules (unsupported).

It was not primarily developed with a cross-compilation support. The
target system is x86-something... The ARM support is something very
new.

> globally tell it not to build modules. Manually maintaining a list of config 
> options to disable is a bit cumbersome.

No, the modules are built based on the configuration, so an entry like
CONFIG_* something.

> 
> 
> >  
> >>  
> >>> 	$(call KCONFIG_DISABLE_OPT,CONFIG_RTE_$(m),$(@D)/build/.config))
> >>> endef
> >>>
> >>> DPDK_POST_CONFIGURE_HOOKS += DPDK_DISABLE_KMODS  
> >>
> >>    Is dpdk using Kconfig? In that case, please use the kconfig-package infra. It
> >> gives you simple primitives for disabling and enabling options. Consult the manual.  
> >
> > Unfortunately, there is no kconfig. It just looks this way. The buildsystem is custom.  
> 
>   So you have to manually create a config file with lines like
> # CONFIG_FOO is not set

Those lines are not used. There is no dependency check...

> ? How uncomfortable...

Not really. But yes ;). The new defconfigs include some core ones
(e.g. common_linuxapp) and just disable what they don't need or
vice-versa...

> 
> >  
> >>  
> >>> endif
> >>>
> >>> Probably, I can improve it by testing whether a module is present
> >>> in the DPDK config and then setting the dependency on linux.  
> >>
> >>    That's not possible: DEPENDENCIES must be declared when the makefiles are
> >> parsed, but the tarball containing .config is only extracted afterwards.  
> >
> > OK, it was just an idea.  
> 
>   We can also still revert to depending on linux unconditionally like you have 
> now. How likely is it that dpdk will be used in practice with no kernel modules? 
> Or is this going to be more likely in the future, with heavier use of UIO and 
> device tree?

I think that the DPDK community is going to use the mainline drivers as
much as possible (vfio). As far as I know, the UIO is considered
broken as it exposes too much into the userspace. Moreover, they
want to be supported by the major distros so anything very custom
is not very good. There are however two modules that are probably not to
be upstreamed: kni (connection between DPDK and the kernel network
stack) and xen (globally disabled by default). I've never used them
myself but others probably do...

Regards
Jan

> 
> 
>   Regards,
>   Arnout
>

Patch
diff mbox

diff --git a/package/Config.in b/package/Config.in
index bcfe13b..022d02f 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1102,6 +1102,7 @@  menu "Networking"
 	source "package/cgic/Config.in"
 	source "package/cppzmq/Config.in"
 	source "package/czmq/Config.in"
+	source "package/dpdk/Config.in"
 	source "package/filemq/Config.in"
 	source "package/flickcurl/Config.in"
 	source "package/fmlib/Config.in"
diff --git a/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch b/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
new file mode 100644
index 0000000..9ac18a5
--- /dev/null
+++ b/package/dpdk/0001-mk-do-not-enforce-any-specific-ARM-ABI.patch
@@ -0,0 +1,31 @@ 
+From 524dc6e45487737f1465e7edade840613f083c8f Mon Sep 17 00:00:00 2001
+From: Jan Viktorin <viktorin@rehivetech.com>
+Date: Sat, 16 Apr 2016 00:11:18 +0200
+Subject: [PATCH] mk: do not enforce any specific ARM ABI
+
+The dpdk build system passes -mfloat-abi=softfp, which makes the build fail
+when the selected ABI is EABIhf. The dpdk build system should not make
+assumptions on the selected ARM ABI.
+
+Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
+Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ mk/machine/armv7a/rte.vars.mk | 2 --
+ 1 file changed, 2 deletions(-)
+
+diff --git a/mk/machine/armv7a/rte.vars.mk b/mk/machine/armv7a/rte.vars.mk
+index abdb15e..36fa3de 100644
+--- a/mk/machine/armv7a/rte.vars.mk
++++ b/mk/machine/armv7a/rte.vars.mk
+@@ -54,8 +54,6 @@
+ # CPU_LDFLAGS =
+ # CPU_ASFLAGS =
+ 
+-CPU_CFLAGS += -mfloat-abi=softfp
+-
+ MACHINE_CFLAGS += -march=armv7-a
+ 
+ ifdef CONFIG_RTE_ARCH_ARM_TUNE
+-- 
+2.8.0
+
diff --git a/package/dpdk/Config.in b/package/dpdk/Config.in
new file mode 100644
index 0000000..27bf106
--- /dev/null
+++ b/package/dpdk/Config.in
@@ -0,0 +1,60 @@ 
+config BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	bool
+	depends on BR2_TOOLCHAIN_HAS_SYNC_1 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_2 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_4 &&\
+		BR2_TOOLCHAIN_HAS_SYNC_8
+	default y if (BR2_i386 && !BR2_x86_i386 && !BR2_x86_i486 \
+			&& !BR2_x86_i586 && !BR2_x86_x1000)
+	default y if BR2_x86_64
+	default y if BR2_ARM_CPU_ARMV7A
+	default y if BR2_aarch64 || BR2_aarch64_be
+
+comment "dpdk needs a glibc toolchain and a Linux Kernel to be built"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_LINUX_KERNEL
+
+config BR2_PACKAGE_DPDK
+	bool "dpdk"
+	depends on BR2_PACKAGE_DPDK_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC
+	depends on BR2_LINUX_KERNEL
+	select BR2_LINUX_NEEDS_MODULES
+	help
+	  DPDK is a set of libraries and drivers for fast packet processing. It
+	  was designed to run on any processors, however, Intel x86 has been
+	  the first CPU to be supported. Ports for other CPUs like IBM Power 8
+	  and ARM are under progress. It runs mostly in Linux userland. A
+	  FreeBSD port is now available for a subset of DPDK features.
+
+	  Notes:
+	  * To build the included Linux Kernel drivers, it is necessary to
+	    enable CONFIG_PCI_MSI, CONFIG_UIO.
+	  * To build the PCAP PMD properly, you need to enable the libpcap
+	    manually.
+	  * You may need to install the python2 interpreter if you want to use
+	    scripts dpdk_nic_bind.py and cpu_layout.py
+
+	  http://www.dpdk.org/
+
+if BR2_PACKAGE_DPDK
+
+config BR2_PACKAGE_DPDK_CONFIG
+	string "Configuration"
+	default "i686-native-linuxapp-gcc" \
+		if BR2_x86_i686
+	default "x86_64-native-linuxapp-gcc" \
+		if BR2_x86_64
+	default "arm-armv7a-linuxapp-gcc" \
+		if BR2_ARM_CPU_ARMV7A
+	default "arm64-armv8a-linuxapp-gcc" \
+		if BR2_aarch64 || BR2_aarch64_be
+
+config BR2_PACKAGE_DPDK_TEST
+	bool "Install tests suite"
+	select BR2_PACKAGE_PYTHON_PEXPECT if BR2_PACKAGE_PYTHON
+	help
+	  Install all DPDK tests. If you want to run the tests by the included
+	  autotest.py script you need to enable python manually.
+
+endif
diff --git a/package/dpdk/dpdk.hash b/package/dpdk/dpdk.hash
new file mode 100644
index 0000000..3780c66
--- /dev/null
+++ b/package/dpdk/dpdk.hash
@@ -0,0 +1,2 @@ 
+# Locally calculated
+sha256	d631495bc6e8d4c4aec72999ac03c3ce213bb996cb88f3bf14bb980dad1d3f7b  dpdk-16.04.tar.gz
diff --git a/package/dpdk/dpdk.mk b/package/dpdk/dpdk.mk
new file mode 100644
index 0000000..ea53417
--- /dev/null
+++ b/package/dpdk/dpdk.mk
@@ -0,0 +1,60 @@ 
+################################################################################
+#
+# dpdk
+#
+################################################################################
+
+DPDK_VERSION = 16.04
+DPDK_SITE = http://dpdk.org/browse/dpdk/snapshot
+
+DPDK_LICENSE = BSD-2c (core), GPLv2+ (Linux drivers)
+DPDK_LICENSE_FILES = GNUmakefile LICENSE.GPL
+DPDK_INSTALL_STAGING = YES
+
+DPDK_DEPENDENCIES += linux
+
+ifeq ($(BR2_PACKAGE_LIBPCAP),y)
+DPDK_DEPENDENCIES += libpcap
+endif
+
+ifeq ($(BR2_SHARED_LIBS),y)
+define DPDK_ENABLE_SHARED_LIBS
+	$(call KCONFIG_ENABLE_OPT,CONFIG_RTE_BUILD_SHARED_LIB,\
+			$(@D)/build/.config)
+endef
+
+DPDK_POST_CONFIGURE_HOOKS += DPDK_ENABLE_SHARED_LIBS
+endif
+
+DPDK_CONFIG = $(call qstrip,$(BR2_PACKAGE_DPDK_CONFIG))
+
+define DPDK_CONFIGURE_CMDS
+	$(MAKE) -C $(@D) T=$(DPDK_CONFIG) RTE_KERNELDIR=$(LINUX_DIR) \
+			   CROSS=$(TARGET_CROSS) config
+endef
+
+define DPDK_BUILD_CMDS
+	$(MAKE) -C $(@D) RTE_KERNELDIR=$(LINUX_DIR) CROSS=$(TARGET_CROSS)
+endef
+
+define DPDK_INSTALL_STAGING_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(STAGING_DIR) prefix=/usr \
+		 CROSS=$(TARGET_CROSS) install-sdk
+endef
+
+ifeq ($(BR2_PACKAGE_DPDK_TEST),y)
+define DPDK_INSTALL_TARGET_TEST
+	mkdir -p $(TARGET_DIR)/usr/dpdk
+	$(INSTALL) -m 0755 -D $(@D)/build/app/test $(TARGET_DIR)/usr/dpdk/test
+	cp -dpfr $(@D)/app/test/*.py $(TARGET_DIR)/usr/dpdk
+endef
+endif
+
+define DPDK_INSTALL_TARGET_CMDS
+	$(MAKE) -C $(@D) DESTDIR=$(TARGET_DIR) CROSS=$(TARGET_CROSS) \
+		kerneldir=/lib/modules/$(LINUX_VERSION_PROBED)/extra/dpdk \
+		prefix=/usr install-runtime install-kmod
+	$(DPDK_INSTALL_TARGET_TEST)
+endef
+
+$(eval $(generic-package))