| Message ID | 20200902180629.29573-1-sunil.pai.g@intel.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [ovs-dev,dpdk-latest,v4] build: Add support for DPDK meson build. | expand |
> Make based build is deprecated in DPDK. Meson based build to be used for > future DPDK releases. > > This updates travis, configure script and documentation for using DPDK Meson > with OVS. > > Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> Thanks Sunil, I've applied this to dpdk-latest. With the deadlines for v1 for DPDK 20.11 being today I think it's prudent to get the dpdk-latest build running again. Regards Ian > --- > v3->v4: > - Fix checkpatch errors > > v2->v3: > - Update Documentation for vhost-user > > v1->v2: > - Update Documentation > - Simplify the pkg-config parsing script > - Rename and move the pkg-config parsing script to python dir > - Update travis to: > - install DPDK to cached dir > - disable DPDK tests > - removed fPIC flag for DPDK > - removed cross compilation for aarch64 > --- > .travis.yml | 3 ++ > .travis/linux-build.sh | 39 ++++++++++----- > .travis/linux-prepare.sh | 1 + > Documentation/intro/install/afxdp.rst | 2 +- > Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > Documentation/topics/dpdk/vhost-user.rst | 18 +------ > acinclude.m4 | 44 +++++++++++------ > python/automake.mk | 3 +- > python/build/pkgcfg.py | 30 +++++++++++ > 9 files changed, 149 insertions(+), 54 deletions(-) create mode 100644 > python/build/pkgcfg.py > > diff --git a/.travis.yml b/.travis.yml > index 3dd5d1d23..a8f9a4d79 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -27,6 +27,9 @@ addons: > - selinux-policy-dev > - libunbound-dev > - libunwind-dev > + - python3-setuptools > + - python3-wheel > + - ninja-build > > before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index > 817bf24aa..14ef833c9 100755 > --- a/.travis/linux-build.sh > +++ b/.travis/linux-build.sh > @@ -85,17 +85,29 @@ function install_dpdk() { > local DPDK_VER=$1 > local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > + local DPDK_OPTS="" > + local DPDK_LIB="" > > if [ -z "$TRAVIS_ARCH" ] || > [ "$TRAVIS_ARCH" == "amd64" ]; then > - TARGET="x86_64-native-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > - TARGET="arm64-armv8a-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > else > echo "Target is unknown" > exit 1 > fi > > + if [ "$DPDK_SHARED" ]; then > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + else > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > + fi > + > + # Export the following path for pkg-config to find the .pc file. > + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > # Avoid using cache for git tree build. > rm -rf dpdk-dir > @@ -108,7 +120,8 @@ function install_dpdk() > if [ -f "${VERSION_FILE}" ]; then > VER=$(cat ${VERSION_FILE}) > if [ "${VER}" = "${DPDK_VER}" ]; then > - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" > + # Update the library paths. > + sudo ldconfig > echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" > return > fi > @@ -122,16 +135,20 @@ function install_dpdk() > pushd dpdk-dir > fi > > - make config CC=gcc T=$TARGET > + # Disable building DPDK unit tests. Not needed for OVS build or tests. > + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > > - if [ "$DPDK_SHARED" ]; then > - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > - fi > + # Install DPDK using prefix. > + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > + > + CC=gcc meson $DPDK_OPTS build > + ninja -C build > + sudo ninja -C build install > + > + # Update the library paths. > + sudo ldconfig > > - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > - echo "Installed DPDK source in $(pwd)" > + echo "Installed DPDK source" > popd > echo "${DPDK_VER}" > ${VERSION_FILE} } diff --git a/.travis/linux-prepare.sh > b/.travis/linux-prepare.sh index 71eb347e8..1baa11641 100755 > --- a/.travis/linux-prepare.sh > +++ b/.travis/linux-prepare.sh > @@ -22,6 +22,7 @@ cd .. > > pip3 install --disable-pip-version-check --user flake8 hacking > pip3 install --user --upgrade docutils > +pip3 install --user 'meson==0.47.1' > > if [ "$M32" ]; then > # Installing 32-bit libraries. > diff --git a/Documentation/intro/install/afxdp.rst > b/Documentation/intro/install/afxdp.rst > index 3c8f78825..327f2b3df 100644 > --- a/Documentation/intro/install/afxdp.rst > +++ b/Documentation/intro/install/afxdp.rst > @@ -396,7 +396,7 @@ PVP using vhostuser device > -------------------------- > First, build OVS with DPDK and AFXDP:: > > - ./configure --enable-afxdp --with-dpdk=<dpdk path> > + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > make -j4 && make install > > Create a vhost-user port from OVS:: > diff --git a/Documentation/intro/install/dpdk.rst > b/Documentation/intro/install/dpdk.rst > index 39544f835..cd7e51c75 100644 > --- a/Documentation/intro/install/dpdk.rst > +++ b/Documentation/intro/install/dpdk.rst > @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > requirements`_. > .. _DPDK supported NIC: http://dpdk.org/doc/nics .. _DPDK requirements: > http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > +.. _dpdk-install: > + > Installing > ---------- > > @@ -76,10 +78,31 @@ Install DPDK > $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > $ cd $DPDK_DIR > > +#. Configure and install DPDK using Meson > + > + Meson is the preferred tool to build recent DPDK releases > + as Make support is deprecated and will be removed from DPDK 20.11. > + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > + > + Build and install the DPDK library:: > + > + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > + $ meson $DPDK_TARGET > + $ ninja -C $DPDK_TARGET > + $ sudo ninja -C $DPDK_TARGET install > + $ sudo ldconfig > + > + Detailed information can be found at `DPDK documentation`_. > + > #. (Optional) Configure DPDK as a shared library > > - DPDK can be built as either a static library or a shared library. By > - default, it is configured for the former. If you wish to use the latter, set > + When using Meson, DPDK is built both as static and shared library. > + So no extra configuration is required in this case. > + > + In case of Make, DPDK can be built as either a static library or a shared > + library. By default, it is configured for the former. If you wish to use > + the latter, set > ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > ``$DPDK_DIR/config/common_base``. > > .. note:: > @@ -87,7 +110,7 @@ Install DPDK > Minor performance loss is expected when using OVS with a shared DPDK > library compared to a static DPDK library. > > -#. Configure and install DPDK > +#. Configure and install DPDK using Make > > Build and install the DPDK library:: > > @@ -97,12 +120,22 @@ Install DPDK > > #. (Optional) Export the DPDK shared library location > > - If DPDK was built as a shared library, export the path to this library for > - use when building OVS:: > + If DPDK was built as a shared library using Make, export the path to this > + library for use when building OVS:: > > $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib > > + In case of Meson, exporting the path to library is not necessary if > + the DPDK libraries are system installed. For libraries installed using > + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this > + library and also update the $PKG_CONFIG_PATH for use before building OVS:: > + > + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > .. _DPDK sources: http://dpdk.org/rel > +.. _DPDK documentation: > +https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > > Install OVS > ~~~~~~~~~~~ > @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library > (``--with-dpdk``). > > #. Bootstrap, if required, as described in :ref:`general-bootstrapping` > > -#. Configure the package using the ``--with-dpdk`` flag:: > +#. Configure the package using the ``--with-dpdk`` flag: > + > + Depending on the tool used to build DPDK and the type of > + DPDK library to use, one can configure OVS as follows: > + > + When DPDK is built using Meson, and OVS must consume DPDK shared > libraries > + (also equivalent to leaving --with-dpdk option empty):: > + > + $ ./configure --with-dpdk=shared > + > + When DPDK is built using Meson, and OVS must consume DPDK static > libraries:: > + > + $ ./configure --with-dpdk=static > + > + When DPDK is built using Make(for shared or static):: > > $ ./configure --with-dpdk=$DPDK_BUILD > > where ``DPDK_BUILD`` is the path to the built DPDK library. This can be > skipped if DPDK library is installed in its default location. > > - If no path is provided to ``--with-dpdk``, but a pkg-config configuration > - for libdpdk is available the include paths will be generated via an > - equivalent ``pkg-config --cflags libdpdk``. > - > .. note:: > While ``--with-dpdk`` is required, you can pass any other configuration > option described in :ref:`general-configuring`. > diff --git a/Documentation/topics/dpdk/vhost-user.rst > b/Documentation/topics/dpdk/vhost-user.rst > index 4af738d11..da3c4c9fa 100644 > --- a/Documentation/topics/dpdk/vhost-user.rst > +++ b/Documentation/topics/dpdk/vhost-user.rst > @@ -389,23 +389,7 @@ application in the VM. > > To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or > :ref:`dpdk-vhost-user-client`. Once started, connect to the VM, download the - > DPDK sources to VM and build DPDK:: > - > - $ cd /root/dpdk/ > - $ wget https://fast.dpdk.org/rel/dpdk-19.11.2.tar.xz > - $ tar xf dpdk-19.11.2.tar.xz > - $ export DPDK_DIR=/root/dpdk/dpdk-stable-19.11.2 > - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > - $ cd $DPDK_DIR > - $ make install T=$DPDK_TARGET DESTDIR=install > - > -Build the test-pmd application:: > - > - $ cd app/test-pmd > - $ export RTE_SDK=$DPDK_DIR > - $ export RTE_TARGET=$DPDK_TARGET > - $ make > +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. > > Setup huge pages and DPDK devices using UIO:: > > diff --git a/acinclude.m4 b/acinclude.m4 index 84f344da0..412b2dd55 100644 > --- a/acinclude.m4 > +++ b/acinclude.m4 > @@ -334,8 +334,10 @@ dnl > dnl Configure DPDK source tree > AC_DEFUN([OVS_CHECK_DPDK], [ > AC_ARG_WITH([dpdk], > - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], > - [Specify the DPDK build directory])], > + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], > + [Specify "static" or "shared" depending on the > + DPDK libraries to use only if built using Meson > + OR the DPDK build directory in case of > + Make])], > [have_dpdk=true]) > > AC_MSG_CHECKING([whether dpdk is enabled]) @@ -345,13 +347,24 @@ > AC_DEFUN([OVS_CHECK_DPDK], [ > else > AC_MSG_RESULT([yes]) > case "$with_dpdk" in > - yes) > + "shared" | "static" | "yes") > DPDK_AUTO_DISCOVER="true" > - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > - DPDK_INCLUDE="$DPDK_CFLAGS" > - DPDK_LIB="$DPDK_LIBS"], [ > - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > - DPDK_LIB="-ldpdk"]) > + case "$with_dpdk" in > + "shared" | "yes") > + PKG_CHECK_MODULES([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + "static") > + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + esac > ;; > *) > DPDK_AUTO_DISCOVER="false" > @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > [AC_MSG_RESULT([no]) > if test "$DPDK_AUTO_DISCOVER" = "true"; then > AC_MSG_ERROR(m4_normalize([ > - Could not find DPDK library in default search path, Use --with-dpdk > - to specify the DPDK library installed in non-standard location])) > + Could not find DPDK library in default search path, update > + PKG_CONFIG_PATH for pkg-config to find the .pc file in > + non-standard location])) > else > AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) > fi > @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > # OTOH newer versions of dpdk pkg-config (generated with Meson) > # will already have flagged just the right set of libs with > # --whole-archive - in those cases do not wrap it once more. > - case "$DPDK_LIB" in > - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; > - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole- > archive > - esac > + if [[ "$pkg_failed" != "no" ]];then > + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole- > archive > + else > + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py > $DPDK_LIB` > + fi > + > AC_SUBST([DPDK_vswitchd_LDFLAGS]) > AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) > fi > diff --git a/python/automake.mk b/python/automake.mk index > 2f08c7701..69d9800f9 100644 > --- a/python/automake.mk > +++ b/python/automake.mk > @@ -47,7 +47,8 @@ ovs_pyfiles = \ > EXTRA_DIST += \ > python/build/__init__.py \ > python/build/nroff.py \ > - python/build/soutil.py > + python/build/soutil.py \ > + python/build/pkgcfg.py > > # PyPI support. > EXTRA_DIST += \ > diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py new file mode > 100644 index 000000000..7cee3cb03 > --- /dev/null > +++ b/python/build/pkgcfg.py > @@ -0,0 +1,30 @@ > +# Copyright (c) 2020 Intel, Inc. > +# > +# Licensed under the Apache License, Version 2.0 (the "License") # You > +may not use this file except in compliance with the License. > +# You may obtain a copy of the License at: > +# > +# http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by applicable law or agreed to in writing, software # > +Distributed under the License is distributed on an "AS IS" BASIS, # > +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > implied. > +# See the License for the specific language governing permissions and # > +Limitations under the License. > + > +# The purpose of this script is to parse the libraries # From > +pkg-config in case of DPDK Meson builds. > + > +import sys > +def parse_pkg_cfg_libs(arg): > + linker_prefix = "-Wl," > + # Libtool expects libraries to be comma separated > + # And -Wl must appear only once. > + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') > + final_string = arg[0]+" "+linker_prefix+final_string > + # Ld only understands -lpthread. > + final_string = final_string.replace('-pthread','-lpthread') > + return final_string > + > +if __name__ == "__main__": > + print(parse_pkg_cfg_libs(sys.argv[1:])) > -- > 2.17.1
On 9/11/20 4:33 PM, Stokes, Ian wrote: >> Make based build is deprecated in DPDK. Meson based build to be used for >> future DPDK releases. >> >> This updates travis, configure script and documentation for using DPDK Meson >> with OVS. >> >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > > Thanks Sunil, I've applied this to dpdk-latest. With the deadlines for v1 for DPDK 20.11 being today I think it's prudent to get the dpdk-latest build running again. Thanks, Ian. I think it's good to have dpdk-latest wokring again. However, this patch will need some changes that will need to be done before moving this to master branch. Could be done as follow up change with squashing before pushing to master. Main point is that, IIUC, this patch changes default build configuration from static linking to dynamic which it should not. And the documentation for --with-dpdk configuration knob is confusing, i.e. it's not aligned with the actual code, If I'm not mistaken. There are few other minor comments too. I'll write them down once I'll find some time. Just wanted to highlight big behavioral change for now. Best regards, Ilya Maximets. > > Regards > Ian >> --- >> v3->v4: >> - Fix checkpatch errors >> >> v2->v3: >> - Update Documentation for vhost-user >> >> v1->v2: >> - Update Documentation >> - Simplify the pkg-config parsing script >> - Rename and move the pkg-config parsing script to python dir >> - Update travis to: >> - install DPDK to cached dir >> - disable DPDK tests >> - removed fPIC flag for DPDK >> - removed cross compilation for aarch64 >> --- >> .travis.yml | 3 ++ >> .travis/linux-build.sh | 39 ++++++++++----- >> .travis/linux-prepare.sh | 1 + >> Documentation/intro/install/afxdp.rst | 2 +- >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ >> acinclude.m4 | 44 +++++++++++------ >> python/automake.mk | 3 +- >> python/build/pkgcfg.py | 30 +++++++++++ >> 9 files changed, 149 insertions(+), 54 deletions(-) create mode 100644 >> python/build/pkgcfg.py >> >> diff --git a/.travis.yml b/.travis.yml >> index 3dd5d1d23..a8f9a4d79 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -27,6 +27,9 @@ addons: >> - selinux-policy-dev >> - libunbound-dev >> - libunwind-dev >> + - python3-setuptools >> + - python3-wheel >> + - ninja-build >> >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >> >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index >> 817bf24aa..14ef833c9 100755 >> --- a/.travis/linux-build.sh >> +++ b/.travis/linux-build.sh >> @@ -85,17 +85,29 @@ function install_dpdk() { >> local DPDK_VER=$1 >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" >> + local DPDK_OPTS="" >> + local DPDK_LIB="" >> >> if [ -z "$TRAVIS_ARCH" ] || >> [ "$TRAVIS_ARCH" == "amd64" ]; then >> - TARGET="x86_64-native-linuxapp-gcc" >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then >> - TARGET="arm64-armv8a-linuxapp-gcc" >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu >> else >> echo "Target is unknown" >> exit 1 >> fi >> >> + if [ "$DPDK_SHARED" ]; then >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >> + else >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" >> + fi >> + >> + # Export the following path for pkg-config to find the .pc file. >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >> + >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then >> # Avoid using cache for git tree build. >> rm -rf dpdk-dir >> @@ -108,7 +120,8 @@ function install_dpdk() >> if [ -f "${VERSION_FILE}" ]; then >> VER=$(cat ${VERSION_FILE}) >> if [ "${VER}" = "${DPDK_VER}" ]; then >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" >> + # Update the library paths. >> + sudo ldconfig >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" >> return >> fi >> @@ -122,16 +135,20 @@ function install_dpdk() >> pushd dpdk-dir >> fi >> >> - make config CC=gcc T=$TARGET >> + # Disable building DPDK unit tests. Not needed for OVS build or tests. >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" >> >> - if [ "$DPDK_SHARED" ]; then >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib >> - fi >> + # Install DPDK using prefix. >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" >> + >> + CC=gcc meson $DPDK_OPTS build >> + ninja -C build >> + sudo ninja -C build install >> + >> + # Update the library paths. >> + sudo ldconfig >> >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" >> - echo "Installed DPDK source in $(pwd)" >> + echo "Installed DPDK source" >> popd >> echo "${DPDK_VER}" > ${VERSION_FILE} } diff --git a/.travis/linux-prepare.sh >> b/.travis/linux-prepare.sh index 71eb347e8..1baa11641 100755 >> --- a/.travis/linux-prepare.sh >> +++ b/.travis/linux-prepare.sh >> @@ -22,6 +22,7 @@ cd .. >> >> pip3 install --disable-pip-version-check --user flake8 hacking >> pip3 install --user --upgrade docutils >> +pip3 install --user 'meson==0.47.1' >> >> if [ "$M32" ]; then >> # Installing 32-bit libraries. >> diff --git a/Documentation/intro/install/afxdp.rst >> b/Documentation/intro/install/afxdp.rst >> index 3c8f78825..327f2b3df 100644 >> --- a/Documentation/intro/install/afxdp.rst >> +++ b/Documentation/intro/install/afxdp.rst >> @@ -396,7 +396,7 @@ PVP using vhostuser device >> -------------------------- >> First, build OVS with DPDK and AFXDP:: >> >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> >> make -j4 && make install >> >> Create a vhost-user port from OVS:: >> diff --git a/Documentation/intro/install/dpdk.rst >> b/Documentation/intro/install/dpdk.rst >> index 39544f835..cd7e51c75 100644 >> --- a/Documentation/intro/install/dpdk.rst >> +++ b/Documentation/intro/install/dpdk.rst >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK >> requirements`_. >> .. _DPDK supported NIC: http://dpdk.org/doc/nics .. _DPDK requirements: >> http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html >> >> +.. _dpdk-install: >> + >> Installing >> ---------- >> >> @@ -76,10 +78,31 @@ Install DPDK >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 >> $ cd $DPDK_DIR >> >> +#. Configure and install DPDK using Meson >> + >> + Meson is the preferred tool to build recent DPDK releases >> + as Make support is deprecated and will be removed from DPDK 20.11. >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. >> + >> + Build and install the DPDK library:: >> + >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >> + $ meson $DPDK_TARGET >> + $ ninja -C $DPDK_TARGET >> + $ sudo ninja -C $DPDK_TARGET install >> + $ sudo ldconfig >> + >> + Detailed information can be found at `DPDK documentation`_. >> + >> #. (Optional) Configure DPDK as a shared library >> >> - DPDK can be built as either a static library or a shared library. By >> - default, it is configured for the former. If you wish to use the latter, set >> + When using Meson, DPDK is built both as static and shared library. >> + So no extra configuration is required in this case. >> + >> + In case of Make, DPDK can be built as either a static library or a shared >> + library. By default, it is configured for the former. If you wish to use >> + the latter, set >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in >> ``$DPDK_DIR/config/common_base``. >> >> .. note:: >> @@ -87,7 +110,7 @@ Install DPDK >> Minor performance loss is expected when using OVS with a shared DPDK >> library compared to a static DPDK library. >> >> -#. Configure and install DPDK >> +#. Configure and install DPDK using Make >> >> Build and install the DPDK library:: >> >> @@ -97,12 +120,22 @@ Install DPDK >> >> #. (Optional) Export the DPDK shared library location >> >> - If DPDK was built as a shared library, export the path to this library for >> - use when building OVS:: >> + If DPDK was built as a shared library using Make, export the path to this >> + library for use when building OVS:: >> >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib >> >> + In case of Meson, exporting the path to library is not necessary if >> + the DPDK libraries are system installed. For libraries installed using >> + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this >> + library and also update the $PKG_CONFIG_PATH for use before building OVS:: >> + >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >> + >> .. _DPDK sources: http://dpdk.org/rel >> +.. _DPDK documentation: >> +https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html >> >> Install OVS >> ~~~~~~~~~~~ >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library >> (``--with-dpdk``). >> >> #. Bootstrap, if required, as described in :ref:`general-bootstrapping` >> >> -#. Configure the package using the ``--with-dpdk`` flag:: >> +#. Configure the package using the ``--with-dpdk`` flag: >> + >> + Depending on the tool used to build DPDK and the type of >> + DPDK library to use, one can configure OVS as follows: >> + >> + When DPDK is built using Meson, and OVS must consume DPDK shared >> libraries >> + (also equivalent to leaving --with-dpdk option empty):: >> + >> + $ ./configure --with-dpdk=shared >> + >> + When DPDK is built using Meson, and OVS must consume DPDK static >> libraries:: >> + >> + $ ./configure --with-dpdk=static >> + >> + When DPDK is built using Make(for shared or static):: >> >> $ ./configure --with-dpdk=$DPDK_BUILD >> >> where ``DPDK_BUILD`` is the path to the built DPDK library. This can be >> skipped if DPDK library is installed in its default location. >> >> - If no path is provided to ``--with-dpdk``, but a pkg-config configuration >> - for libdpdk is available the include paths will be generated via an >> - equivalent ``pkg-config --cflags libdpdk``. >> - >> .. note:: >> While ``--with-dpdk`` is required, you can pass any other configuration >> option described in :ref:`general-configuring`. >> diff --git a/Documentation/topics/dpdk/vhost-user.rst >> b/Documentation/topics/dpdk/vhost-user.rst >> index 4af738d11..da3c4c9fa 100644 >> --- a/Documentation/topics/dpdk/vhost-user.rst >> +++ b/Documentation/topics/dpdk/vhost-user.rst >> @@ -389,23 +389,7 @@ application in the VM. >> >> To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or >> :ref:`dpdk-vhost-user-client`. Once started, connect to the VM, download the - >> DPDK sources to VM and build DPDK:: >> - >> - $ cd /root/dpdk/ >> - $ wget https://fast.dpdk.org/rel/dpdk-19.11.2.tar.xz >> - $ tar xf dpdk-19.11.2.tar.xz >> - $ export DPDK_DIR=/root/dpdk/dpdk-stable-19.11.2 >> - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >> - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >> - $ cd $DPDK_DIR >> - $ make install T=$DPDK_TARGET DESTDIR=install >> - >> -Build the test-pmd application:: >> - >> - $ cd app/test-pmd >> - $ export RTE_SDK=$DPDK_DIR >> - $ export RTE_TARGET=$DPDK_TARGET >> - $ make >> +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. >> >> Setup huge pages and DPDK devices using UIO:: >> >> diff --git a/acinclude.m4 b/acinclude.m4 index 84f344da0..412b2dd55 100644 >> --- a/acinclude.m4 >> +++ b/acinclude.m4 >> @@ -334,8 +334,10 @@ dnl >> dnl Configure DPDK source tree >> AC_DEFUN([OVS_CHECK_DPDK], [ >> AC_ARG_WITH([dpdk], >> - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], >> - [Specify the DPDK build directory])], >> + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], >> + [Specify "static" or "shared" depending on the >> + DPDK libraries to use only if built using Meson >> + OR the DPDK build directory in case of >> + Make])], >> [have_dpdk=true]) >> >> AC_MSG_CHECKING([whether dpdk is enabled]) @@ -345,13 +347,24 @@ >> AC_DEFUN([OVS_CHECK_DPDK], [ >> else >> AC_MSG_RESULT([yes]) >> case "$with_dpdk" in >> - yes) >> + "shared" | "static" | "yes") >> DPDK_AUTO_DISCOVER="true" >> - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ >> - DPDK_INCLUDE="$DPDK_CFLAGS" >> - DPDK_LIB="$DPDK_LIBS"], [ >> - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" >> - DPDK_LIB="-ldpdk"]) >> + case "$with_dpdk" in >> + "shared" | "yes") >> + PKG_CHECK_MODULES([DPDK], [libdpdk], [ >> + DPDK_INCLUDE="$DPDK_CFLAGS" >> + DPDK_LIB="$DPDK_LIBS"], [ >> + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" >> + DPDK_LIB="-ldpdk"]) >> + ;; >> + "static") >> + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ >> + DPDK_INCLUDE="$DPDK_CFLAGS" >> + DPDK_LIB="$DPDK_LIBS"], [ >> + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" >> + DPDK_LIB="-ldpdk"]) >> + ;; >> + esac >> ;; >> *) >> DPDK_AUTO_DISCOVER="false" >> @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ >> [AC_MSG_RESULT([no]) >> if test "$DPDK_AUTO_DISCOVER" = "true"; then >> AC_MSG_ERROR(m4_normalize([ >> - Could not find DPDK library in default search path, Use --with-dpdk >> - to specify the DPDK library installed in non-standard location])) >> + Could not find DPDK library in default search path, update >> + PKG_CONFIG_PATH for pkg-config to find the .pc file in >> + non-standard location])) >> else >> AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) >> fi >> @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ >> # OTOH newer versions of dpdk pkg-config (generated with Meson) >> # will already have flagged just the right set of libs with >> # --whole-archive - in those cases do not wrap it once more. >> - case "$DPDK_LIB" in >> - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; >> - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole- >> archive >> - esac >> + if [[ "$pkg_failed" != "no" ]];then >> + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole- >> archive >> + else >> + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py >> $DPDK_LIB` >> + fi >> + >> AC_SUBST([DPDK_vswitchd_LDFLAGS]) >> AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) >> fi >> diff --git a/python/automake.mk b/python/automake.mk index >> 2f08c7701..69d9800f9 100644 >> --- a/python/automake.mk >> +++ b/python/automake.mk >> @@ -47,7 +47,8 @@ ovs_pyfiles = \ >> EXTRA_DIST += \ >> python/build/__init__.py \ >> python/build/nroff.py \ >> - python/build/soutil.py >> + python/build/soutil.py \ >> + python/build/pkgcfg.py >> >> # PyPI support. >> EXTRA_DIST += \ >> diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py new file mode >> 100644 index 000000000..7cee3cb03 >> --- /dev/null >> +++ b/python/build/pkgcfg.py >> @@ -0,0 +1,30 @@ >> +# Copyright (c) 2020 Intel, Inc. >> +# >> +# Licensed under the Apache License, Version 2.0 (the "License") # You >> +may not use this file except in compliance with the License. >> +# You may obtain a copy of the License at: >> +# >> +# http://www.apache.org/licenses/LICENSE-2.0 >> +# >> +# Unless required by applicable law or agreed to in writing, software # >> +Distributed under the License is distributed on an "AS IS" BASIS, # >> +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >> implied. >> +# See the License for the specific language governing permissions and # >> +Limitations under the License. >> + >> +# The purpose of this script is to parse the libraries # From >> +pkg-config in case of DPDK Meson builds. >> + >> +import sys >> +def parse_pkg_cfg_libs(arg): >> + linker_prefix = "-Wl," >> + # Libtool expects libraries to be comma separated >> + # And -Wl must appear only once. >> + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') >> + final_string = arg[0]+" "+linker_prefix+final_string >> + # Ld only understands -lpthread. >> + final_string = final_string.replace('-pthread','-lpthread') >> + return final_string >> + >> +if __name__ == "__main__": >> + print(parse_pkg_cfg_libs(sys.argv[1:])) >> -- >> 2.17.1 >
> -----Original Message----- > From: Ilya Maximets <i.maximets@ovn.org> > Sent: Friday, September 11, 2020 8:14 PM > To: Stokes, Ian <ian.stokes@intel.com>; Pai G, Sunil <sunil.pai.g@intel.com>; > dev@openvswitch.org > Cc: i.maximets@ovn.org; david.marchand@redhat.com; Richardson, Bruce > <bruce.richardson@intel.com>; christian.ehrhardt@canonical.com; > i.maximets@ovn.org > Subject: Re: [PATCH dpdk-latest v4] build: Add support for DPDK meson > build. > > On 9/11/20 4:33 PM, Stokes, Ian wrote: > >> Make based build is deprecated in DPDK. Meson based build to be used > >> for future DPDK releases. > >> > >> This updates travis, configure script and documentation for using > >> DPDK Meson with OVS. > >> > >> Tested-at: > >> https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > > > > Thanks Sunil, I've applied this to dpdk-latest. With the deadlines for v1 for > DPDK 20.11 being today I think it's prudent to get the dpdk-latest build > running again. > > Thanks, Ian. > > I think it's good to have dpdk-latest wokring again. > > However, this patch will need some changes that will need to be done > before moving this to master branch. Could be done as follow up change > with squashing before pushing to master. > > Main point is that, IIUC, this patch changes default build configuration from > static linking to dynamic which it should not. > Hi Ilya , Before this change , when using DPDK meson build with OVS , I saw that DPDK shared libraries are used by default even when asked for static because of the --whole-archive and --no-whole-archive flags getting jumbled by the libtool. I wasn’t sure if I had to change this behavior, hence retained it. :) I will send out another patch addressing this along with the other comments on this patch when they are available. > And the documentation for --with-dpdk configuration knob is confusing, i.e. > it's not aligned with the actual code, If I'm not mistaken. > I would be happy to make it as simple as possible. Suggestions are welcome :) > There are few other minor comments too. I'll write them down once I'll find > some time. Just wanted to highlight big behavioral change for now. Sure , will wait on your feedback. <snipped> Thanks and regards Sunil
On 9/2/20 8:06 PM, Sunil Pai G wrote: > Make based build is deprecated in DPDK. Meson based > build to be used for future DPDK releases. > > This updates travis, configure script and documentation > for using DPDK Meson with OVS. > > Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > --- Sorry for delay. Following up with things that I think should be fixed before this patch merged to master. > v3->v4: > - Fix checkpatch errors > > v2->v3: > - Update Documentation for vhost-user > > v1->v2: > - Update Documentation > - Simplify the pkg-config parsing script > - Rename and move the pkg-config parsing script to python dir > - Update travis to: > - install DPDK to cached dir > - disable DPDK tests > - removed fPIC flag for DPDK > - removed cross compilation for aarch64 > --- > .travis.yml | 3 ++ > .travis/linux-build.sh | 39 ++++++++++----- > .travis/linux-prepare.sh | 1 + > Documentation/intro/install/afxdp.rst | 2 +- > Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > Documentation/topics/dpdk/vhost-user.rst | 18 +------ > acinclude.m4 | 44 +++++++++++------ > python/automake.mk | 3 +- > python/build/pkgcfg.py | 30 +++++++++++ > 9 files changed, 149 insertions(+), 54 deletions(-) > create mode 100644 python/build/pkgcfg.py > > diff --git a/.travis.yml b/.travis.yml > index 3dd5d1d23..a8f9a4d79 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -27,6 +27,9 @@ addons: > - selinux-policy-dev > - libunbound-dev > - libunwind-dev > + - python3-setuptools > + - python3-wheel > + - ninja-build > > before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > index 817bf24aa..14ef833c9 100755 > --- a/.travis/linux-build.sh > +++ b/.travis/linux-build.sh > @@ -85,17 +85,29 @@ function install_dpdk() > { > local DPDK_VER=$1 > local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > + local DPDK_OPTS="" > + local DPDK_LIB="" > > if [ -z "$TRAVIS_ARCH" ] || > [ "$TRAVIS_ARCH" == "amd64" ]; then > - TARGET="x86_64-native-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > - TARGET="arm64-armv8a-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > else > echo "Target is unknown" > exit 1 > fi > > + if [ "$DPDK_SHARED" ]; then > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + else > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > + fi > + > + # Export the following path for pkg-config to find the .pc file. > + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > # Avoid using cache for git tree build. > rm -rf dpdk-dir > @@ -108,7 +120,8 @@ function install_dpdk() > if [ -f "${VERSION_FILE}" ]; then > VER=$(cat ${VERSION_FILE}) > if [ "${VER}" = "${DPDK_VER}" ]; then > - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" > + # Update the library paths. > + sudo ldconfig > echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" > return > fi > @@ -122,16 +135,20 @@ function install_dpdk() > pushd dpdk-dir > fi > > - make config CC=gcc T=$TARGET > + # Disable building DPDK unit tests. Not needed for OVS build or tests. > + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > > - if [ "$DPDK_SHARED" ]; then > - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > - fi > + # Install DPDK using prefix. > + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > + > + CC=gcc meson $DPDK_OPTS build > + ninja -C build > + sudo ninja -C build install Why we need 'sudo' here? We're installing to a local folder, should work without 'sudo', IIUC. > + > + # Update the library paths. > + sudo ldconfig > > - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > - echo "Installed DPDK source in $(pwd)" > + echo "Installed DPDK source" It's probably better to keep this line as is. > popd > echo "${DPDK_VER}" > ${VERSION_FILE} > } > diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > index 71eb347e8..1baa11641 100755 > --- a/.travis/linux-prepare.sh > +++ b/.travis/linux-prepare.sh > @@ -22,6 +22,7 @@ cd .. > > pip3 install --disable-pip-version-check --user flake8 hacking > pip3 install --user --upgrade docutils > +pip3 install --user 'meson==0.47.1' > > if [ "$M32" ]; then > # Installing 32-bit libraries. > diff --git a/Documentation/intro/install/afxdp.rst b/Documentation/intro/install/afxdp.rst > index 3c8f78825..327f2b3df 100644 > --- a/Documentation/intro/install/afxdp.rst > +++ b/Documentation/intro/install/afxdp.rst > @@ -396,7 +396,7 @@ PVP using vhostuser device > -------------------------- > First, build OVS with DPDK and AFXDP:: > > - ./configure --enable-afxdp --with-dpdk=<dpdk path> > + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> There should be no '<dpdk path>' option. See below. > make -j4 && make install > > Create a vhost-user port from OVS:: > diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst > index 39544f835..cd7e51c75 100644 > --- a/Documentation/intro/install/dpdk.rst > +++ b/Documentation/intro/install/dpdk.rst > @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK requirements`_. > .. _DPDK supported NIC: http://dpdk.org/doc/nics > .. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > +.. _dpdk-install: > + > Installing > ---------- > > @@ -76,10 +78,31 @@ Install DPDK > $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > $ cd $DPDK_DIR > > +#. Configure and install DPDK using Meson > + > + Meson is the preferred tool to build recent DPDK releases > + as Make support is deprecated and will be removed from DPDK 20.11. > + OVS supports DPDK Meson builds from DPDK 19.11 onwards. Do we support meson builds with 19.11? I'm not sure that it works, otherwise why we need this patch? We support builds with DPDK libs provided by pkg-config, but that doesn't mean that we support build with meson. IIRC, it's not possible to link agains DPDK built with meson unless you're crafting a combined library by hands. I can't name this as 'support'. > + > + Build and install the DPDK library:: > + > + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > + $ meson $DPDK_TARGET This looks confusing. Makes an impression that this actually affects target you build for. But that is just a name of a folder and build will be for your current architecture with a default compiler. We should not use word 'TARGET' here as it is misleading. > + $ ninja -C $DPDK_TARGET > + $ sudo ninja -C $DPDK_TARGET install > + $ sudo ldconfig > + > + Detailed information can be found at `DPDK documentation`_. > + > #. (Optional) Configure DPDK as a shared library > > - DPDK can be built as either a static library or a shared library. By > - default, it is configured for the former. If you wish to use the latter, set > + When using Meson, DPDK is built both as static and shared library. > + So no extra configuration is required in this case. > + > + In case of Make, DPDK can be built as either a static library or a shared > + library. By default, it is configured for the former. If you wish to use > + the latter, set > ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in ``$DPDK_DIR/config/common_base``. 'make' build system is no longe a way to build DPDK. We should remove support of build with 'make' along with documentation that describes how to do that. > > .. note:: > @@ -87,7 +110,7 @@ Install DPDK > Minor performance loss is expected when using OVS with a shared DPDK > library compared to a static DPDK library. > > -#. Configure and install DPDK > +#. Configure and install DPDK using Make Same point. This section of a document should just be removed. > > Build and install the DPDK library:: > > @@ -97,12 +120,22 @@ Install DPDK > > #. (Optional) Export the DPDK shared library location > > - If DPDK was built as a shared library, export the path to this library for > - use when building OVS:: > + If DPDK was built as a shared library using Make, export the path to this > + library for use when building OVS:: > > $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib > > + In case of Meson, exporting the path to library is not necessary if > + the DPDK libraries are system installed. For libraries installed using > + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this > + library and also update the $PKG_CONFIG_PATH for use before building OVS:: > + > + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > .. _DPDK sources: http://dpdk.org/rel > +.. _DPDK documentation: https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html This should be versioned link, i.e. link to 20.11 version of docs. > > Install OVS > ~~~~~~~~~~~ > @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library (``--with-dpdk``). > > #. Bootstrap, if required, as described in :ref:`general-bootstrapping` > > -#. Configure the package using the ``--with-dpdk`` flag:: > +#. Configure the package using the ``--with-dpdk`` flag: > + > + Depending on the tool used to build DPDK and the type of > + DPDK library to use, one can configure OVS as follows: > + > + When DPDK is built using Meson, and OVS must consume DPDK shared libraries > + (also equivalent to leaving --with-dpdk option empty):: Default option should be 'static', i.e. empty option should mean 'static'. Anyway, code below has a switch between 'static', 'shared' and 'yes'. There is no empty option. > + > + $ ./configure --with-dpdk=shared > + > + When DPDK is built using Meson, and OVS must consume DPDK static libraries:: > + > + $ ./configure --with-dpdk=static > + > + When DPDK is built using Make(for shared or static):: > > $ ./configure --with-dpdk=$DPDK_BUILD > > where ``DPDK_BUILD`` is the path to the built DPDK library. This can be > skipped if DPDK library is installed in its default location. > > - If no path is provided to ``--with-dpdk``, but a pkg-config configuration > - for libdpdk is available the include paths will be generated via an > - equivalent ``pkg-config --cflags libdpdk``. > - > .. note:: > While ``--with-dpdk`` is required, you can pass any other configuration > option described in :ref:`general-configuring`. > diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst > index 4af738d11..da3c4c9fa 100644 > --- a/Documentation/topics/dpdk/vhost-user.rst > +++ b/Documentation/topics/dpdk/vhost-user.rst > @@ -389,23 +389,7 @@ application in the VM. > > To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or > :ref:`dpdk-vhost-user-client`. Once started, connect to the VM, download the > -DPDK sources to VM and build DPDK:: > - > - $ cd /root/dpdk/ > - $ wget https://fast.dpdk.org/rel/dpdk-19.11.2.tar.xz > - $ tar xf dpdk-19.11.2.tar.xz > - $ export DPDK_DIR=/root/dpdk/dpdk-stable-19.11.2 > - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > - $ cd $DPDK_DIR > - $ make install T=$DPDK_TARGET DESTDIR=install > - > -Build the test-pmd application:: > - > - $ cd app/test-pmd > - $ export RTE_SDK=$DPDK_DIR > - $ export RTE_TARGET=$DPDK_TARGET > - $ make > +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. > > Setup huge pages and DPDK devices using UIO:: > > diff --git a/acinclude.m4 b/acinclude.m4 > index 84f344da0..412b2dd55 100644 > --- a/acinclude.m4 > +++ b/acinclude.m4 > @@ -334,8 +334,10 @@ dnl > dnl Configure DPDK source tree > AC_DEFUN([OVS_CHECK_DPDK], [ > AC_ARG_WITH([dpdk], > - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], > - [Specify the DPDK build directory])], > + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], static|shared|yes > + [Specify "static" or "shared" depending on the > + DPDK libraries to use only if built using Meson > + OR the DPDK build directory in case of Make])], > [have_dpdk=true]) > > AC_MSG_CHECKING([whether dpdk is enabled]) > @@ -345,13 +347,24 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > else > AC_MSG_RESULT([yes]) > case "$with_dpdk" in > - yes) > + "shared" | "static" | "yes") > DPDK_AUTO_DISCOVER="true" > - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > - DPDK_INCLUDE="$DPDK_CFLAGS" > - DPDK_LIB="$DPDK_LIBS"], [ > - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > - DPDK_LIB="-ldpdk"]) > + case "$with_dpdk" in > + "shared" | "yes") "yes" should mean "static" as it was before. See PKG_CHECK_MODULES_STATIC() above. > + PKG_CHECK_MODULES([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + "static") > + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + esac > ;; > *) > DPDK_AUTO_DISCOVER="false" All the 'DPDK_AUTO_DISCOVER="false"'-related code should be removed along with support for 'make' builds. > @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > [AC_MSG_RESULT([no]) > if test "$DPDK_AUTO_DISCOVER" = "true"; then > AC_MSG_ERROR(m4_normalize([ > - Could not find DPDK library in default search path, Use --with-dpdk > - to specify the DPDK library installed in non-standard location])) > + Could not find DPDK library in default search path, update > + PKG_CONFIG_PATH for pkg-config to find the .pc file in > + non-standard location])) > else > AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) > fi > @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > # OTOH newer versions of dpdk pkg-config (generated with Meson) > # will already have flagged just the right set of libs with > # --whole-archive - in those cases do not wrap it once more. At least part of the above comment is no more valid. > - case "$DPDK_LIB" in > - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; > - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive > - esac > + if [[ "$pkg_failed" != "no" ]];then > + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive > + else > + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py $DPDK_LIB` > + fi > + > AC_SUBST([DPDK_vswitchd_LDFLAGS]) > AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) > fi > diff --git a/python/automake.mk b/python/automake.mk > index 2f08c7701..69d9800f9 100644 > --- a/python/automake.mk > +++ b/python/automake.mk > @@ -47,7 +47,8 @@ ovs_pyfiles = \ > EXTRA_DIST += \ > python/build/__init__.py \ > python/build/nroff.py \ > - python/build/soutil.py > + python/build/soutil.py \ > + python/build/pkgcfg.py Should be added to FLAKE8_PYFILES. This will likely expose some PEP8 issues that should be fixed. > > # PyPI support. > EXTRA_DIST += \ > diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py > new file mode 100644 > index 000000000..7cee3cb03 > --- /dev/null > +++ b/python/build/pkgcfg.py > @@ -0,0 +1,30 @@ shebang is missing. > +# Copyright (c) 2020 Intel, Inc. > +# > +# Licensed under the Apache License, Version 2.0 (the "License") > +# You may not use this file except in compliance with the License. > +# You may obtain a copy of the License at: > +# > +# http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by applicable law or agreed to in writing, software > +# Distributed under the License is distributed on an "AS IS" BASIS, > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > +# See the License for the specific language governing permissions and > +# Limitations under the License. > + > +# The purpose of this script is to parse the libraries > +# From pkg-config in case of DPDK Meson builds. What exactly this script does and why it's needed? More extensive comment required. Probably, with a short example. Maybe it makes sense to move comment from acinclude.m4 here and keep only the short description there. > + > +import sys > +def parse_pkg_cfg_libs(arg): > + linker_prefix = "-Wl," > + # Libtool expects libraries to be comma separated > + # And -Wl must appear only once. > + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') > + final_string = arg[0]+" "+linker_prefix+final_string > + # Ld only understands -lpthread. > + final_string = final_string.replace('-pthread','-lpthread') > + return final_string > + > +if __name__ == "__main__": > + print(parse_pkg_cfg_libs(sys.argv[1:])) > In general, please, add some spaces after commas and around binary operators. Best regards, Ilya Maximets.
Hi Ilya , Thank you for the comments , please see response inline. > -----Original Message----- > From: Ilya Maximets <i.maximets@ovn.org> > Sent: Tuesday, November 10, 2020 2:27 AM > To: Pai G, Sunil <sunil.pai.g@intel.com>; dev@openvswitch.org > Cc: Stokes, Ian <ian.stokes@intel.com>; i.maximets@ovn.org; > david.marchand@redhat.com; Richardson, Bruce > <bruce.richardson@intel.com>; christian.ehrhardt@canonical.com; > i.maximets@ovn.org; Kevin Traynor <ktraynor@redhat.com> > Subject: Re: [PATCH dpdk-latest v4] build: Add support for DPDK meson > build. > > On 9/2/20 8:06 PM, Sunil Pai G wrote: > > Make based build is deprecated in DPDK. Meson based build to be used > > for future DPDK releases. > > > > This updates travis, configure script and documentation for using DPDK > > Meson with OVS. > > > > Tested-at: > > https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > > Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > > --- > > Sorry for delay. Following up with things that I think should be fixed before > this patch merged to master. Not a problem :) > > > v3->v4: > > - Fix checkpatch errors > > > > v2->v3: > > - Update Documentation for vhost-user > > > > v1->v2: > > - Update Documentation > > - Simplify the pkg-config parsing script > > - Rename and move the pkg-config parsing script to python dir > > - Update travis to: > > - install DPDK to cached dir > > - disable DPDK tests > > - removed fPIC flag for DPDK > > - removed cross compilation for aarch64 > > --- > > .travis.yml | 3 ++ > > .travis/linux-build.sh | 39 ++++++++++----- > > .travis/linux-prepare.sh | 1 + > > Documentation/intro/install/afxdp.rst | 2 +- > > Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > > Documentation/topics/dpdk/vhost-user.rst | 18 +------ > > acinclude.m4 | 44 +++++++++++------ > > python/automake.mk | 3 +- > > python/build/pkgcfg.py | 30 +++++++++++ > > 9 files changed, 149 insertions(+), 54 deletions(-) create mode > > 100644 python/build/pkgcfg.py > > > > diff --git a/.travis.yml b/.travis.yml index 3dd5d1d23..a8f9a4d79 > > 100644 > > --- a/.travis.yml > > +++ b/.travis.yml > > @@ -27,6 +27,9 @@ addons: > > - selinux-policy-dev > > - libunbound-dev > > - libunwind-dev > > + - python3-setuptools > > + - python3-wheel > > + - ninja-build > > > > <snipped> > > + CC=gcc meson $DPDK_OPTS build > > + ninja -C build > > + sudo ninja -C build install > > Why we need 'sudo' here? We're installing to a local folder, should work > without 'sudo', IIUC. Sure , Will try removing the sudo. > > > + > > + # Update the library paths. > > + sudo ldconfig > > > > - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > > - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > > - echo "Installed DPDK source in $(pwd)" > > + echo "Installed DPDK source" > > It's probably better to keep this line as is. Sure. > > > popd > > echo "${DPDK_VER}" > ${VERSION_FILE} } diff --git > > a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh index > > 71eb347e8..1baa11641 100755 > > --- a/.travis/linux-prepare.sh > > +++ b/.travis/linux-prepare.sh > > @@ -22,6 +22,7 @@ cd .. > > > > pip3 install --disable-pip-version-check --user flake8 hacking > > pip3 install --user --upgrade docutils > > +pip3 install --user 'meson==0.47.1' > > > > if [ "$M32" ]; then > > # Installing 32-bit libraries. > > diff --git a/Documentation/intro/install/afxdp.rst > > b/Documentation/intro/install/afxdp.rst > > index 3c8f78825..327f2b3df 100644 > > --- a/Documentation/intro/install/afxdp.rst > > +++ b/Documentation/intro/install/afxdp.rst > > @@ -396,7 +396,7 @@ PVP using vhostuser device > > -------------------------- > > First, build OVS with DPDK and AFXDP:: > > > > - ./configure --enable-afxdp --with-dpdk=<dpdk path> > > + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > > There should be no '<dpdk path>' option. See below. Sure . > > > make -j4 && make install > > > > Create a vhost-user port from OVS:: > > diff --git a/Documentation/intro/install/dpdk.rst > > b/Documentation/intro/install/dpdk.rst > > index 39544f835..cd7e51c75 100644 > > --- a/Documentation/intro/install/dpdk.rst > > +++ b/Documentation/intro/install/dpdk.rst > > @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > requirements`_. > > .. _DPDK supported NIC: http://dpdk.org/doc/nics .. _DPDK > > requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > > > +.. _dpdk-install: > > + > > Installing > > ---------- > > > > @@ -76,10 +78,31 @@ Install DPDK > > $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > > $ cd $DPDK_DIR > > > > +#. Configure and install DPDK using Meson > > + > > + Meson is the preferred tool to build recent DPDK releases > > + as Make support is deprecated and will be removed from DPDK 20.11. > > + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > > Do we support meson builds with 19.11? I'm not sure that it works, > otherwise why we need this patch? We do, but it doesn’t work as expected. Christian had added the initial support. > > We support builds with DPDK libs provided by pkg-config, but that doesn't > mean that we support build with meson. IIRC, it's not possible to link agains > DPDK built with meson unless you're crafting a combined library by hands. I > can't name this as 'support'. > Sure . > > + > > + Build and install the DPDK library:: > > + > > + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > > + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > > + $ meson $DPDK_TARGET > > This looks confusing. Makes an impression that this actually affects target > you build for. But that is just a name of a folder and build will be for your > current architecture with a default compiler. > We should not use word 'TARGET' here as it is misleading. > Sure , let me remove the DPDK_TARGET and have something like below: $ export DPDK_BUILD=$DPDK_DIR/build > > + $ ninja -C $DPDK_TARGET > > + $ sudo ninja -C $DPDK_TARGET install > > + $ sudo ldconfig > > + > > + Detailed information can be found at `DPDK documentation`_. > > + > > #. (Optional) Configure DPDK as a shared library > > > > - DPDK can be built as either a static library or a shared library. By > > - default, it is configured for the former. If you wish to use the latter, set > > + When using Meson, DPDK is built both as static and shared library. > > + So no extra configuration is required in this case. > > + > > + In case of Make, DPDK can be built as either a static library or a shared > > + library. By default, it is configured for the former. If you wish to use > > + the latter, set > > ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > ``$DPDK_DIR/config/common_base``. > > 'make' build system is no longe a way to build DPDK. We should remove > support of build with 'make' along with documentation that describes how to > do that. > Sure , will remove make documentation. > > > > .. note:: > > @@ -87,7 +110,7 @@ Install DPDK > > Minor performance loss is expected when using OVS with a shared > DPDK > > library compared to a static DPDK library. > > > > -#. Configure and install DPDK > > +#. Configure and install DPDK using Make > > Same point. This section of a document should just be removed. Sure. > > > > > Build and install the DPDK library:: > > > > @@ -97,12 +120,22 @@ Install DPDK > > > > #. (Optional) Export the DPDK shared library location > > > > - If DPDK was built as a shared library, export the path to this library for > > - use when building OVS:: > > + If DPDK was built as a shared library using Make, export the path to this > > + library for use when building OVS:: > > > > $ export > > LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib > > > > + In case of Meson, exporting the path to library is not necessary if > > + the DPDK libraries are system installed. For libraries installed using > > + a prefix(assuming $DPDK_INSTALL in the below case), export the path to > this > > + library and also update the $PKG_CONFIG_PATH for use before building > OVS:: > > + > > + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > > + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > > + $ export > PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > > + > > .. _DPDK sources: http://dpdk.org/rel > > +.. _DPDK documentation: > > +https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > > This should be versioned link, i.e. link to 20.11 version of docs. Sure , will link to 20.11 version. > > > > > Install OVS > > ~~~~~~~~~~~ > > @@ -121,17 +154,27 @@ has to be configured to build against the DPDK > library (``--with-dpdk``). > > > > #. Bootstrap, if required, as described in > > :ref:`general-bootstrapping` > > > > -#. Configure the package using the ``--with-dpdk`` flag:: > > +#. Configure the package using the ``--with-dpdk`` flag: > > + > > + Depending on the tool used to build DPDK and the type of > > + DPDK library to use, one can configure OVS as follows: > > + > > + When DPDK is built using Meson, and OVS must consume DPDK shared > libraries > > + (also equivalent to leaving --with-dpdk option empty):: > > Default option should be 'static', i.e. empty option should mean 'static'. > Anyway, code below has a switch between 'static', 'shared' and 'yes'. There > is no empty option. The empty option defaults to "yes" is what I found IIRC. Regarding the default option being static , Christian's initial support had a bug which was causing the DPDK shared libraries to be used instead of static ones though the intention was to use the static DPDK libs. This was because of the libtool re arranging the --whole-archive and --no-whole-archive flags. In order to fix this, I added the pkgcfg.py file below. I didn’t want to break this existing behavior although faulty. I wouldn’t mind changing it to static. Let me know your thoughts. > > > + > > + $ ./configure --with-dpdk=shared > > + > > + When DPDK is built using Meson, and OVS must consume DPDK static > libraries:: > > + > > + $ ./configure --with-dpdk=static > > + <snipped> > > +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. > > > > Setup huge pages and DPDK devices using UIO:: > > > > diff --git a/acinclude.m4 b/acinclude.m4 index 84f344da0..412b2dd55 > > 100644 > > --- a/acinclude.m4 > > +++ b/acinclude.m4 > > @@ -334,8 +334,10 @@ dnl > > dnl Configure DPDK source tree > > AC_DEFUN([OVS_CHECK_DPDK], [ > > AC_ARG_WITH([dpdk], > > - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], > > - [Specify the DPDK build directory])], > > + > > + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], > > static|shared|yes Sure > > > + [Specify "static" or "shared" depending on the > > + DPDK libraries to use only if built using Meson > > + OR the DPDK build directory in case of > > + Make])], > > [have_dpdk=true]) > > > > AC_MSG_CHECKING([whether dpdk is enabled]) @@ -345,13 +347,24 > @@ > > AC_DEFUN([OVS_CHECK_DPDK], [ > > else > > AC_MSG_RESULT([yes]) > > case "$with_dpdk" in > > - yes) > > + "shared" | "static" | "yes") > > DPDK_AUTO_DISCOVER="true" > > - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > > - DPDK_INCLUDE="$DPDK_CFLAGS" > > - DPDK_LIB="$DPDK_LIBS"], [ > > - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > > - DPDK_LIB="-ldpdk"]) > > + case "$with_dpdk" in > > + "shared" | "yes") > > "yes" should mean "static" as it was before. See > PKG_CHECK_MODULES_STATIC() above. Sure. > > > + PKG_CHECK_MODULES([DPDK], [libdpdk], [ > > + DPDK_INCLUDE="$DPDK_CFLAGS" > > + DPDK_LIB="$DPDK_LIBS"], [ > > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > > + DPDK_LIB="-ldpdk"]) > > + ;; > > + "static") > > + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > > + DPDK_INCLUDE="$DPDK_CFLAGS" > > + DPDK_LIB="$DPDK_LIBS"], [ > > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > > + DPDK_LIB="-ldpdk"]) > > + ;; > > + esac > > ;; > > *) > > DPDK_AUTO_DISCOVER="false" > > All the 'DPDK_AUTO_DISCOVER="false"'-related code should be removed > along with support for 'make' builds. Sure > > > @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > > [AC_MSG_RESULT([no]) > > if test "$DPDK_AUTO_DISCOVER" = "true"; then > > AC_MSG_ERROR(m4_normalize([ > > - Could not find DPDK library in default search path, Use --with-dpdk > > - to specify the DPDK library installed in non-standard location])) > > + Could not find DPDK library in default search path, update > > + PKG_CONFIG_PATH for pkg-config to find the .pc file in > > + non-standard location])) > > else > > AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) > > fi > > @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > > # OTOH newer versions of dpdk pkg-config (generated with Meson) > > # will already have flagged just the right set of libs with > > # --whole-archive - in those cases do not wrap it once more. > > At least part of the above comment is no more valid. Sure , will update the comment. > > > - case "$DPDK_LIB" in > > - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; > > - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no- > whole-archive > > - esac > > + if [[ "$pkg_failed" != "no" ]];then > > + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no- > whole-archive > > + else > > + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py > $DPDK_LIB` > > + fi > > + > > AC_SUBST([DPDK_vswitchd_LDFLAGS]) > > AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) > > fi > > diff --git a/python/automake.mk b/python/automake.mk index > > 2f08c7701..69d9800f9 100644 > > --- a/python/automake.mk > > +++ b/python/automake.mk > > @@ -47,7 +47,8 @@ ovs_pyfiles = \ > > EXTRA_DIST += \ > > python/build/__init__.py \ > > python/build/nroff.py \ > > - python/build/soutil.py > > + python/build/soutil.py \ > > + python/build/pkgcfg.py > > Should be added to FLAKE8_PYFILES. This will likely expose some PEP8 issues > that should be fixed. Sure , will add it to FLAKE8_PYFILES. > > > > > # PyPI support. > > EXTRA_DIST += \ > > diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py new file > > mode 100644 index 000000000..7cee3cb03 > > --- /dev/null > > +++ b/python/build/pkgcfg.py > > @@ -0,0 +1,30 @@ > > shebang is missing. Sure , will add it . > > > +# Copyright (c) 2020 Intel, Inc. > > +# > > +# Licensed under the Apache License, Version 2.0 (the "License") # > > +You may not use this file except in compliance with the License. > > +# You may obtain a copy of the License at: > > +# > > +# http://www.apache.org/licenses/LICENSE-2.0 > > +# > > +# Unless required by applicable law or agreed to in writing, software > > +# Distributed under the License is distributed on an "AS IS" BASIS, # > > +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > implied. > > +# See the License for the specific language governing permissions and > > +# Limitations under the License. > > + > > +# The purpose of this script is to parse the libraries # From > > +pkg-config in case of DPDK Meson builds. > > What exactly this script does and why it's needed? > More extensive comment required. Probably, with a short example. > Maybe it makes sense to move comment from acinclude.m4 here and keep > only the short description there. This goes back to the libtool re-arranging the --whole-archive and --no-whole-archive flags. Libtool seems to pack the linker flags together causing : -L/usr/local/lib/x86_64-linux-gnu -Wl,--whole-archive,<libs>, -Wl,--no-whole-archive, <libs> to appear as : ... -Wl,--whole-archive -Wl,--no-whole-archive ... which removes the effect of these linker flags altogether So , I added this script , which keeps only one entry for -Wl, so that it doesn’t get jumbled. This was also probably why even incase of make , we had something like this in the acinclude.m4 , where -Wl was only provided only once. "DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive" Will add these comments in the file as well. > > > + > > +import sys > > +def parse_pkg_cfg_libs(arg): > > + linker_prefix = "-Wl," > > + # Libtool expects libraries to be comma separated > > + # And -Wl must appear only once. > > + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') > > + final_string = arg[0]+" "+linker_prefix+final_string > > + # Ld only understands -lpthread. > > + final_string = final_string.replace('-pthread','-lpthread') > > + return final_string > > + > > +if __name__ == "__main__": > > + print(parse_pkg_cfg_libs(sys.argv[1:])) > > > > In general, please, add some spaces after commas and around binary > operators. Sure , will address this. Thanks and regards, Sunil Pai
On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> wrote: > > Make based build is deprecated in DPDK. Meson based > build to be used for future DPDK releases. > > This updates travis, configure script and documentation > for using DPDK Meson with OVS. > > Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > --- > v3->v4: > - Fix checkpatch errors > > v2->v3: > - Update Documentation for vhost-user > > v1->v2: > - Update Documentation > - Simplify the pkg-config parsing script > - Rename and move the pkg-config parsing script to python dir > - Update travis to: > - install DPDK to cached dir > - disable DPDK tests > - removed fPIC flag for DPDK > - removed cross compilation for aarch64 > --- > .travis.yml | 3 ++ > .travis/linux-build.sh | 39 ++++++++++----- > .travis/linux-prepare.sh | 1 + > Documentation/intro/install/afxdp.rst | 2 +- > Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > Documentation/topics/dpdk/vhost-user.rst | 18 +------ > acinclude.m4 | 44 +++++++++++------ > python/automake.mk | 3 +- > python/build/pkgcfg.py | 30 +++++++++++ > 9 files changed, 149 insertions(+), 54 deletions(-) > create mode 100644 python/build/pkgcfg.py > > diff --git a/.travis.yml b/.travis.yml > index 3dd5d1d23..a8f9a4d79 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -27,6 +27,9 @@ addons: > - selinux-policy-dev > - libunbound-dev > - libunwind-dev > + - python3-setuptools > + - python3-wheel > + - ninja-build > > before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > index 817bf24aa..14ef833c9 100755 > --- a/.travis/linux-build.sh > +++ b/.travis/linux-build.sh > @@ -85,17 +85,29 @@ function install_dpdk() > { > local DPDK_VER=$1 > local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > + local DPDK_OPTS="" > + local DPDK_LIB="" > > if [ -z "$TRAVIS_ARCH" ] || > [ "$TRAVIS_ARCH" == "amd64" ]; then > - TARGET="x86_64-native-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > - TARGET="arm64-armv8a-linuxapp-gcc" > + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > else > echo "Target is unknown" > exit 1 > fi > > + if [ "$DPDK_SHARED" ]; then > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + else > + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > + fi > + > + # Export the following path for pkg-config to find the .pc file. > + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > # Avoid using cache for git tree build. > rm -rf dpdk-dir > @@ -108,7 +120,8 @@ function install_dpdk() > if [ -f "${VERSION_FILE}" ]; then > VER=$(cat ${VERSION_FILE}) > if [ "${VER}" = "${DPDK_VER}" ]; then > - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" > + # Update the library paths. > + sudo ldconfig > echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" > return > fi > @@ -122,16 +135,20 @@ function install_dpdk() > pushd dpdk-dir > fi > > - make config CC=gcc T=$TARGET > + # Disable building DPDK unit tests. Not needed for OVS build or tests. > + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > > - if [ "$DPDK_SHARED" ]; then > - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > - fi > + # Install DPDK using prefix. > + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > + > + CC=gcc meson $DPDK_OPTS build > + ninja -C build > + sudo ninja -C build install > + > + # Update the library paths. > + sudo ldconfig > > - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > - echo "Installed DPDK source in $(pwd)" > + echo "Installed DPDK source" > popd > echo "${DPDK_VER}" > ${VERSION_FILE} > } > diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > index 71eb347e8..1baa11641 100755 > --- a/.travis/linux-prepare.sh > +++ b/.travis/linux-prepare.sh > @@ -22,6 +22,7 @@ cd .. > > pip3 install --disable-pip-version-check --user flake8 hacking > pip3 install --user --upgrade docutils > +pip3 install --user 'meson==0.47.1' > > if [ "$M32" ]; then > # Installing 32-bit libraries. > diff --git a/Documentation/intro/install/afxdp.rst b/Documentation/intro/install/afxdp.rst > index 3c8f78825..327f2b3df 100644 > --- a/Documentation/intro/install/afxdp.rst > +++ b/Documentation/intro/install/afxdp.rst > @@ -396,7 +396,7 @@ PVP using vhostuser device > -------------------------- > First, build OVS with DPDK and AFXDP:: > > - ./configure --enable-afxdp --with-dpdk=<dpdk path> > + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > make -j4 && make install > > Create a vhost-user port from OVS:: > diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst > index 39544f835..cd7e51c75 100644 > --- a/Documentation/intro/install/dpdk.rst > +++ b/Documentation/intro/install/dpdk.rst > @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK requirements`_. > .. _DPDK supported NIC: http://dpdk.org/doc/nics > .. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > +.. _dpdk-install: > + > Installing > ---------- > > @@ -76,10 +78,31 @@ Install DPDK > $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > $ cd $DPDK_DIR > > +#. Configure and install DPDK using Meson > + > + Meson is the preferred tool to build recent DPDK releases > + as Make support is deprecated and will be removed from DPDK 20.11. > + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > + > + Build and install the DPDK library:: > + > + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > + $ meson $DPDK_TARGET > + $ ninja -C $DPDK_TARGET > + $ sudo ninja -C $DPDK_TARGET install > + $ sudo ldconfig > + > + Detailed information can be found at `DPDK documentation`_. > + > #. (Optional) Configure DPDK as a shared library > > - DPDK can be built as either a static library or a shared library. By > - default, it is configured for the former. If you wish to use the latter, set > + When using Meson, DPDK is built both as static and shared library. > + So no extra configuration is required in this case. > + > + In case of Make, DPDK can be built as either a static library or a shared > + library. By default, it is configured for the former. If you wish to use > + the latter, set > ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in ``$DPDK_DIR/config/common_base``. > > .. note:: > @@ -87,7 +110,7 @@ Install DPDK > Minor performance loss is expected when using OVS with a shared DPDK > library compared to a static DPDK library. > > -#. Configure and install DPDK > +#. Configure and install DPDK using Make > > Build and install the DPDK library:: > > @@ -97,12 +120,22 @@ Install DPDK > > #. (Optional) Export the DPDK shared library location > > - If DPDK was built as a shared library, export the path to this library for > - use when building OVS:: > + If DPDK was built as a shared library using Make, export the path to this > + library for use when building OVS:: > > $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib > > + In case of Meson, exporting the path to library is not necessary if > + the DPDK libraries are system installed. For libraries installed using > + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this > + library and also update the $PKG_CONFIG_PATH for use before building OVS:: > + > + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > + > .. _DPDK sources: http://dpdk.org/rel > +.. _DPDK documentation: https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > > Install OVS > ~~~~~~~~~~~ > @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library (``--with-dpdk``). > > #. Bootstrap, if required, as described in :ref:`general-bootstrapping` > > -#. Configure the package using the ``--with-dpdk`` flag:: > +#. Configure the package using the ``--with-dpdk`` flag: > + > + Depending on the tool used to build DPDK and the type of > + DPDK library to use, one can configure OVS as follows: > + > + When DPDK is built using Meson, and OVS must consume DPDK shared libraries > + (also equivalent to leaving --with-dpdk option empty):: > + > + $ ./configure --with-dpdk=shared Hi all when I build ovs with --with-dpdk=shared, Compilation is fine, but we I launch the ovs: 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 --socket-limit 1024,1024. 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device "0000:82:00.0" 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: No such device In dpdk: eal_plugins_init rte_pci_scan rte_bus_register eal_option_device_parse -- no pci bus pkg-config is pkgconf-1.4.2 > + When DPDK is built using Meson, and OVS must consume DPDK static libraries:: > + > + $ ./configure --with-dpdk=static Only one issue, if we use the pkgconfig-0.27 The ovs log: 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit 1024,1024. 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device "0000:82:00.0" 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No such device Because /bin/pkg-config --static --libs libdpdk pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' Note that: -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't work.) I guess we dont hope that, It should be: -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive update pkg-config to pkgconf-1.4.2 /bin/pkg-config --static --libs libdpdk pkg_cv_DPDK_LIBS='-Wl,--whole-archive -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' ovs works fine. > + When DPDK is built using Make(for shared or static):: > > $ ./configure --with-dpdk=$DPDK_BUILD > > where ``DPDK_BUILD`` is the path to the built DPDK library. This can be > skipped if DPDK library is installed in its default location. > > - If no path is provided to ``--with-dpdk``, but a pkg-config configuration > - for libdpdk is available the include paths will be generated via an > - equivalent ``pkg-config --cflags libdpdk``. > - > .. note:: > While ``--with-dpdk`` is required, you can pass any other configuration > option described in :ref:`general-configuring`. > diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst > index 4af738d11..da3c4c9fa 100644 > --- a/Documentation/topics/dpdk/vhost-user.rst > +++ b/Documentation/topics/dpdk/vhost-user.rst > @@ -389,23 +389,7 @@ application in the VM. > > To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or > :ref:`dpdk-vhost-user-client`. Once started, connect to the VM, download the > -DPDK sources to VM and build DPDK:: > - > - $ cd /root/dpdk/ > - $ wget https://fast.dpdk.org/rel/dpdk-19.11.2.tar.xz > - $ tar xf dpdk-19.11.2.tar.xz > - $ export DPDK_DIR=/root/dpdk/dpdk-stable-19.11.2 > - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > - $ cd $DPDK_DIR > - $ make install T=$DPDK_TARGET DESTDIR=install > - > -Build the test-pmd application:: > - > - $ cd app/test-pmd > - $ export RTE_SDK=$DPDK_DIR > - $ export RTE_TARGET=$DPDK_TARGET > - $ make > +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. > > Setup huge pages and DPDK devices using UIO:: > > diff --git a/acinclude.m4 b/acinclude.m4 > index 84f344da0..412b2dd55 100644 > --- a/acinclude.m4 > +++ b/acinclude.m4 > @@ -334,8 +334,10 @@ dnl > dnl Configure DPDK source tree > AC_DEFUN([OVS_CHECK_DPDK], [ > AC_ARG_WITH([dpdk], > - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], > - [Specify the DPDK build directory])], > + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], > + [Specify "static" or "shared" depending on the > + DPDK libraries to use only if built using Meson > + OR the DPDK build directory in case of Make])], > [have_dpdk=true]) > > AC_MSG_CHECKING([whether dpdk is enabled]) > @@ -345,13 +347,24 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > else > AC_MSG_RESULT([yes]) > case "$with_dpdk" in > - yes) > + "shared" | "static" | "yes") > DPDK_AUTO_DISCOVER="true" > - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > - DPDK_INCLUDE="$DPDK_CFLAGS" > - DPDK_LIB="$DPDK_LIBS"], [ > - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > - DPDK_LIB="-ldpdk"]) > + case "$with_dpdk" in > + "shared" | "yes") > + PKG_CHECK_MODULES([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + "static") > + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ > + DPDK_INCLUDE="$DPDK_CFLAGS" > + DPDK_LIB="$DPDK_LIBS"], [ > + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" > + DPDK_LIB="-ldpdk"]) > + ;; > + esac > ;; > *) > DPDK_AUTO_DISCOVER="false" > @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > [AC_MSG_RESULT([no]) > if test "$DPDK_AUTO_DISCOVER" = "true"; then > AC_MSG_ERROR(m4_normalize([ > - Could not find DPDK library in default search path, Use --with-dpdk > - to specify the DPDK library installed in non-standard location])) > + Could not find DPDK library in default search path, update > + PKG_CONFIG_PATH for pkg-config to find the .pc file in > + non-standard location])) > else > AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) > fi > @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ > # OTOH newer versions of dpdk pkg-config (generated with Meson) > # will already have flagged just the right set of libs with > # --whole-archive - in those cases do not wrap it once more. > - case "$DPDK_LIB" in > - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; > - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive > - esac > + if [[ "$pkg_failed" != "no" ]];then > + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive > + else > + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py $DPDK_LIB` > + fi > + > AC_SUBST([DPDK_vswitchd_LDFLAGS]) > AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) > fi > diff --git a/python/automake.mk b/python/automake.mk > index 2f08c7701..69d9800f9 100644 > --- a/python/automake.mk > +++ b/python/automake.mk > @@ -47,7 +47,8 @@ ovs_pyfiles = \ > EXTRA_DIST += \ > python/build/__init__.py \ > python/build/nroff.py \ > - python/build/soutil.py > + python/build/soutil.py \ > + python/build/pkgcfg.py > > # PyPI support. > EXTRA_DIST += \ > diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py > new file mode 100644 > index 000000000..7cee3cb03 > --- /dev/null > +++ b/python/build/pkgcfg.py > @@ -0,0 +1,30 @@ > +# Copyright (c) 2020 Intel, Inc. > +# > +# Licensed under the Apache License, Version 2.0 (the "License") > +# You may not use this file except in compliance with the License. > +# You may obtain a copy of the License at: > +# > +# http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by applicable law or agreed to in writing, software > +# Distributed under the License is distributed on an "AS IS" BASIS, > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > +# See the License for the specific language governing permissions and > +# Limitations under the License. > + > +# The purpose of this script is to parse the libraries > +# From pkg-config in case of DPDK Meson builds. > + > +import sys > +def parse_pkg_cfg_libs(arg): > + linker_prefix = "-Wl," > + # Libtool expects libraries to be comma separated > + # And -Wl must appear only once. > + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') > + final_string = arg[0]+" "+linker_prefix+final_string > + # Ld only understands -lpthread. > + final_string = final_string.replace('-pthread','-lpthread') > + return final_string > + > +if __name__ == "__main__": > + print(parse_pkg_cfg_libs(sys.argv[1:])) > -- > 2.17.1 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
On 12/15/20 5:02 AM, Tonghao Zhang wrote: > On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> wrote: >> >> Make based build is deprecated in DPDK. Meson based >> build to be used for future DPDK releases. >> >> This updates travis, configure script and documentation >> for using DPDK Meson with OVS. >> >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> >> --- >> v3->v4: >> - Fix checkpatch errors >> >> v2->v3: >> - Update Documentation for vhost-user >> >> v1->v2: >> - Update Documentation >> - Simplify the pkg-config parsing script >> - Rename and move the pkg-config parsing script to python dir >> - Update travis to: >> - install DPDK to cached dir >> - disable DPDK tests >> - removed fPIC flag for DPDK >> - removed cross compilation for aarch64 >> --- >> .travis.yml | 3 ++ >> .travis/linux-build.sh | 39 ++++++++++----- >> .travis/linux-prepare.sh | 1 + >> Documentation/intro/install/afxdp.rst | 2 +- >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ >> acinclude.m4 | 44 +++++++++++------ >> python/automake.mk | 3 +- >> python/build/pkgcfg.py | 30 +++++++++++ >> 9 files changed, 149 insertions(+), 54 deletions(-) >> create mode 100644 python/build/pkgcfg.py >> >> diff --git a/.travis.yml b/.travis.yml >> index 3dd5d1d23..a8f9a4d79 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -27,6 +27,9 @@ addons: >> - selinux-policy-dev >> - libunbound-dev >> - libunwind-dev >> + - python3-setuptools >> + - python3-wheel >> + - ninja-build >> >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >> >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >> index 817bf24aa..14ef833c9 100755 >> --- a/.travis/linux-build.sh >> +++ b/.travis/linux-build.sh >> @@ -85,17 +85,29 @@ function install_dpdk() >> { >> local DPDK_VER=$1 >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" >> + local DPDK_OPTS="" >> + local DPDK_LIB="" >> >> if [ -z "$TRAVIS_ARCH" ] || >> [ "$TRAVIS_ARCH" == "amd64" ]; then >> - TARGET="x86_64-native-linuxapp-gcc" >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then >> - TARGET="arm64-armv8a-linuxapp-gcc" >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu >> else >> echo "Target is unknown" >> exit 1 >> fi >> >> + if [ "$DPDK_SHARED" ]; then >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >> + else >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" >> + fi >> + >> + # Export the following path for pkg-config to find the .pc file. >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >> + >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then >> # Avoid using cache for git tree build. >> rm -rf dpdk-dir >> @@ -108,7 +120,8 @@ function install_dpdk() >> if [ -f "${VERSION_FILE}" ]; then >> VER=$(cat ${VERSION_FILE}) >> if [ "${VER}" = "${DPDK_VER}" ]; then >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" >> + # Update the library paths. >> + sudo ldconfig >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" >> return >> fi >> @@ -122,16 +135,20 @@ function install_dpdk() >> pushd dpdk-dir >> fi >> >> - make config CC=gcc T=$TARGET >> + # Disable building DPDK unit tests. Not needed for OVS build or tests. >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" >> >> - if [ "$DPDK_SHARED" ]; then >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib >> - fi >> + # Install DPDK using prefix. >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" >> + >> + CC=gcc meson $DPDK_OPTS build >> + ninja -C build >> + sudo ninja -C build install >> + >> + # Update the library paths. >> + sudo ldconfig >> >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" >> - echo "Installed DPDK source in $(pwd)" >> + echo "Installed DPDK source" >> popd >> echo "${DPDK_VER}" > ${VERSION_FILE} >> } >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >> index 71eb347e8..1baa11641 100755 >> --- a/.travis/linux-prepare.sh >> +++ b/.travis/linux-prepare.sh >> @@ -22,6 +22,7 @@ cd .. >> >> pip3 install --disable-pip-version-check --user flake8 hacking >> pip3 install --user --upgrade docutils >> +pip3 install --user 'meson==0.47.1' >> >> if [ "$M32" ]; then >> # Installing 32-bit libraries. >> diff --git a/Documentation/intro/install/afxdp.rst b/Documentation/intro/install/afxdp.rst >> index 3c8f78825..327f2b3df 100644 >> --- a/Documentation/intro/install/afxdp.rst >> +++ b/Documentation/intro/install/afxdp.rst >> @@ -396,7 +396,7 @@ PVP using vhostuser device >> -------------------------- >> First, build OVS with DPDK and AFXDP:: >> >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> >> make -j4 && make install >> >> Create a vhost-user port from OVS:: >> diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst >> index 39544f835..cd7e51c75 100644 >> --- a/Documentation/intro/install/dpdk.rst >> +++ b/Documentation/intro/install/dpdk.rst >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK requirements`_. >> .. _DPDK supported NIC: http://dpdk.org/doc/nics >> .. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html >> >> +.. _dpdk-install: >> + >> Installing >> ---------- >> >> @@ -76,10 +78,31 @@ Install DPDK >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 >> $ cd $DPDK_DIR >> >> +#. Configure and install DPDK using Meson >> + >> + Meson is the preferred tool to build recent DPDK releases >> + as Make support is deprecated and will be removed from DPDK 20.11. >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. >> + >> + Build and install the DPDK library:: >> + >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >> + $ meson $DPDK_TARGET >> + $ ninja -C $DPDK_TARGET >> + $ sudo ninja -C $DPDK_TARGET install >> + $ sudo ldconfig >> + >> + Detailed information can be found at `DPDK documentation`_. >> + >> #. (Optional) Configure DPDK as a shared library >> >> - DPDK can be built as either a static library or a shared library. By >> - default, it is configured for the former. If you wish to use the latter, set >> + When using Meson, DPDK is built both as static and shared library. >> + So no extra configuration is required in this case. >> + >> + In case of Make, DPDK can be built as either a static library or a shared >> + library. By default, it is configured for the former. If you wish to use >> + the latter, set >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in ``$DPDK_DIR/config/common_base``. >> >> .. note:: >> @@ -87,7 +110,7 @@ Install DPDK >> Minor performance loss is expected when using OVS with a shared DPDK >> library compared to a static DPDK library. >> >> -#. Configure and install DPDK >> +#. Configure and install DPDK using Make >> >> Build and install the DPDK library:: >> >> @@ -97,12 +120,22 @@ Install DPDK >> >> #. (Optional) Export the DPDK shared library location >> >> - If DPDK was built as a shared library, export the path to this library for >> - use when building OVS:: >> + If DPDK was built as a shared library using Make, export the path to this >> + library for use when building OVS:: >> >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib >> >> + In case of Meson, exporting the path to library is not necessary if >> + the DPDK libraries are system installed. For libraries installed using >> + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this >> + library and also update the $PKG_CONFIG_PATH for use before building OVS:: >> + >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >> + >> .. _DPDK sources: http://dpdk.org/rel >> +.. _DPDK documentation: https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html >> >> Install OVS >> ~~~~~~~~~~~ >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library (``--with-dpdk``). >> >> #. Bootstrap, if required, as described in :ref:`general-bootstrapping` >> >> -#. Configure the package using the ``--with-dpdk`` flag:: >> +#. Configure the package using the ``--with-dpdk`` flag: >> + >> + Depending on the tool used to build DPDK and the type of >> + DPDK library to use, one can configure OVS as follows: >> + >> + When DPDK is built using Meson, and OVS must consume DPDK shared libraries >> + (also equivalent to leaving --with-dpdk option empty):: >> + >> + $ ./configure --with-dpdk=shared > Hi all > when I build ovs with --with-dpdk=shared, Compilation is fine, but we > I launch the ovs: > > 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] > -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > --socket-limit 1024,1024. > 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > "0000:82:00.0" > 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' > 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: > No such device > > In dpdk: > eal_plugins_init > rte_pci_scan > rte_bus_register > eal_option_device_parse -- no pci bus > > pkg-config is pkgconf-1.4.2 >> + When DPDK is built using Meson, and OVS must consume DPDK static libraries:: >> + >> + $ ./configure --with-dpdk=static > Only one issue, if we use the pkgconfig-0.27 > The ovs log: > 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > 1024,1024. > 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > "0000:82:00.0" > 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' > 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No > such device > > Because /bin/pkg-config --static --libs libdpdk > pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > -l:librte_common_cpt.a -l:librte_common_dpaax.a > -l:librte_common_iavf.a -l:librte_common_octeontx.a > -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > -l:librte_common_mlx5.a -l:librte_common_qat.a > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > > Note that: > -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't work.) > I guess we dont hope that, It should be: > -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > update pkg-config to pkgconf-1.4.2 > /bin/pkg-config --static --libs libdpdk > pkg_cv_DPDK_LIBS='-Wl,--whole-archive > -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > -l:librte_common_dpaax.a -l:librte_common_iavf.a > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > ovs works fine. Hmm. Thanks for the report. It'll be great if you can test this with new version of this patch: https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1-git-send-email-ian.stokes@intel.com/ Bruce, David, it looks like not all versions of pkg-config could be used. Do you know what is the minimal required version or how to protect us from this kind of issues? Best regards, Ilya Maximets.
> -----Original Message----- > From: Ilya Maximets <i.maximets@ovn.org> > Sent: Tuesday, December 15, 2020 9:33 AM > To: Tonghao Zhang <xiangxia.m.yue@gmail.com>; Pai G, Sunil > <sunil.pai.g@intel.com> > Cc: ovs dev <dev@openvswitch.org>; Richardson, Bruce > <bruce.richardson@intel.com>; Ilya Maximets <i.maximets@ovn.org>; David > Marchand <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > On 12/15/20 5:02 AM, Tonghao Zhang wrote: > > On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> > wrote: > >> > >> Make based build is deprecated in DPDK. Meson based > >> build to be used for future DPDK releases. > >> > >> This updates travis, configure script and documentation > >> for using DPDK Meson with OVS. > >> > >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs- > copy/builds/723510063 > >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > >> --- > >> v3->v4: > >> - Fix checkpatch errors > >> > >> v2->v3: > >> - Update Documentation for vhost-user > >> > >> v1->v2: > >> - Update Documentation > >> - Simplify the pkg-config parsing script > >> - Rename and move the pkg-config parsing script to python dir > >> - Update travis to: > >> - install DPDK to cached dir > >> - disable DPDK tests > >> - removed fPIC flag for DPDK > >> - removed cross compilation for aarch64 > >> --- > >> .travis.yml | 3 ++ > >> .travis/linux-build.sh | 39 ++++++++++----- > >> .travis/linux-prepare.sh | 1 + > >> Documentation/intro/install/afxdp.rst | 2 +- > >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ > >> acinclude.m4 | 44 +++++++++++------ > >> python/automake.mk | 3 +- > >> python/build/pkgcfg.py | 30 +++++++++++ > >> 9 files changed, 149 insertions(+), 54 deletions(-) > >> create mode 100644 python/build/pkgcfg.py > >> > >> diff --git a/.travis.yml b/.travis.yml > >> index 3dd5d1d23..a8f9a4d79 100644 > >> --- a/.travis.yml > >> +++ b/.travis.yml > >> @@ -27,6 +27,9 @@ addons: > >> - selinux-policy-dev > >> - libunbound-dev > >> - libunwind-dev > >> + - python3-setuptools > >> + - python3-wheel > >> + - ninja-build > >> > >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > >> > >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > >> index 817bf24aa..14ef833c9 100755 > >> --- a/.travis/linux-build.sh > >> +++ b/.travis/linux-build.sh > >> @@ -85,17 +85,29 @@ function install_dpdk() > >> { > >> local DPDK_VER=$1 > >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > >> + local DPDK_OPTS="" > >> + local DPDK_LIB="" > >> > >> if [ -z "$TRAVIS_ARCH" ] || > >> [ "$TRAVIS_ARCH" == "amd64" ]; then > >> - TARGET="x86_64-native-linuxapp-gcc" > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > >> - TARGET="arm64-armv8a-linuxapp-gcc" > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > >> else > >> echo "Target is unknown" > >> exit 1 > >> fi > >> > >> + if [ "$DPDK_SHARED" ]; then > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >> + else > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > >> + fi > >> + > >> + # Export the following path for pkg-config to find the .pc file. > >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >> + > >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > >> # Avoid using cache for git tree build. > >> rm -rf dpdk-dir > >> @@ -108,7 +120,8 @@ function install_dpdk() > >> if [ -f "${VERSION_FILE}" ]; then > >> VER=$(cat ${VERSION_FILE}) > >> if [ "${VER}" = "${DPDK_VER}" ]; then > >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk- > dir/build" > >> + # Update the library paths. > >> + sudo ldconfig > >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk- > dir" > >> return > >> fi > >> @@ -122,16 +135,20 @@ function install_dpdk() > >> pushd dpdk-dir > >> fi > >> > >> - make config CC=gcc T=$TARGET > >> + # Disable building DPDK unit tests. Not needed for OVS build or > tests. > >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > >> > >> - if [ "$DPDK_SHARED" ]; then > >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > >> - fi > >> + # Install DPDK using prefix. > >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > >> + > >> + CC=gcc meson $DPDK_OPTS build > >> + ninja -C build > >> + sudo ninja -C build install > >> + > >> + # Update the library paths. > >> + sudo ldconfig > >> > >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > >> - echo "Installed DPDK source in $(pwd)" > >> + echo "Installed DPDK source" > >> popd > >> echo "${DPDK_VER}" > ${VERSION_FILE} > >> } > >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > >> index 71eb347e8..1baa11641 100755 > >> --- a/.travis/linux-prepare.sh > >> +++ b/.travis/linux-prepare.sh > >> @@ -22,6 +22,7 @@ cd .. > >> > >> pip3 install --disable-pip-version-check --user flake8 hacking > >> pip3 install --user --upgrade docutils > >> +pip3 install --user 'meson==0.47.1' > >> > >> if [ "$M32" ]; then > >> # Installing 32-bit libraries. > >> diff --git a/Documentation/intro/install/afxdp.rst > b/Documentation/intro/install/afxdp.rst > >> index 3c8f78825..327f2b3df 100644 > >> --- a/Documentation/intro/install/afxdp.rst > >> +++ b/Documentation/intro/install/afxdp.rst > >> @@ -396,7 +396,7 @@ PVP using vhostuser device > >> -------------------------- > >> First, build OVS with DPDK and AFXDP:: > >> > >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> > >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > >> make -j4 && make install > >> > >> Create a vhost-user port from OVS:: > >> diff --git a/Documentation/intro/install/dpdk.rst > b/Documentation/intro/install/dpdk.rst > >> index 39544f835..cd7e51c75 100644 > >> --- a/Documentation/intro/install/dpdk.rst > >> +++ b/Documentation/intro/install/dpdk.rst > >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > requirements`_. > >> .. _DPDK supported NIC: http://dpdk.org/doc/nics > >> .. _DPDK requirements: > http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > >> > >> +.. _dpdk-install: > >> + > >> Installing > >> ---------- > >> > >> @@ -76,10 +78,31 @@ Install DPDK > >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > >> $ cd $DPDK_DIR > >> > >> +#. Configure and install DPDK using Meson > >> + > >> + Meson is the preferred tool to build recent DPDK releases > >> + as Make support is deprecated and will be removed from DPDK 20.11. > >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > >> + > >> + Build and install the DPDK library:: > >> + > >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > >> + $ meson $DPDK_TARGET > >> + $ ninja -C $DPDK_TARGET > >> + $ sudo ninja -C $DPDK_TARGET install > >> + $ sudo ldconfig > >> + > >> + Detailed information can be found at `DPDK documentation`_. > >> + > >> #. (Optional) Configure DPDK as a shared library > >> > >> - DPDK can be built as either a static library or a shared library. > By > >> - default, it is configured for the former. If you wish to use the > latter, set > >> + When using Meson, DPDK is built both as static and shared library. > >> + So no extra configuration is required in this case. > >> + > >> + In case of Make, DPDK can be built as either a static library or a > shared > >> + library. By default, it is configured for the former. If you wish > to use > >> + the latter, set > >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > ``$DPDK_DIR/config/common_base``. > >> > >> .. note:: > >> @@ -87,7 +110,7 @@ Install DPDK > >> Minor performance loss is expected when using OVS with a shared > DPDK > >> library compared to a static DPDK library. > >> > >> -#. Configure and install DPDK > >> +#. Configure and install DPDK using Make > >> > >> Build and install the DPDK library:: > >> > >> @@ -97,12 +120,22 @@ Install DPDK > >> > >> #. (Optional) Export the DPDK shared library location > >> > >> - If DPDK was built as a shared library, export the path to this > library for > >> - use when building OVS:: > >> + If DPDK was built as a shared library using Make, export the path > to this > >> + library for use when building OVS:: > >> > >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp- > gcc/lib > >> > >> + In case of Meson, exporting the path to library is not necessary if > >> + the DPDK libraries are system installed. For libraries installed > using > >> + a prefix(assuming $DPDK_INSTALL in the below case), export the path > to this > >> + library and also update the $PKG_CONFIG_PATH for use before > building OVS:: > >> + > >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >> + > >> .. _DPDK sources: http://dpdk.org/rel > >> +.. _DPDK documentation: > https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > >> > >> Install OVS > >> ~~~~~~~~~~~ > >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK > library (``--with-dpdk``). > >> > >> #. Bootstrap, if required, as described in :ref:`general- > bootstrapping` > >> > >> -#. Configure the package using the ``--with-dpdk`` flag:: > >> +#. Configure the package using the ``--with-dpdk`` flag: > >> + > >> + Depending on the tool used to build DPDK and the type of > >> + DPDK library to use, one can configure OVS as follows: > >> + > >> + When DPDK is built using Meson, and OVS must consume DPDK shared > libraries > >> + (also equivalent to leaving --with-dpdk option empty):: > >> + > >> + $ ./configure --with-dpdk=shared > > Hi all > > when I build ovs with --with-dpdk=shared, Compilation is fine, but we > > I launch the ovs: > > > > 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > tor=[0-6] > > -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > --socket-limit 1024,1024. > > 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > ntor=[0-6]' > > 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: > > No such device > > > > In dpdk: > > eal_plugins_init > > rte_pci_scan > > rte_bus_register > > eal_option_device_parse -- no pci bus > > > > pkg-config is pkgconf-1.4.2 > >> + When DPDK is built using Meson, and OVS must consume DPDK static > libraries:: > >> + > >> + $ ./configure --with-dpdk=static > > Only one issue, if we use the pkgconfig-0.27 > > The ovs log: > > 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > tor=[0-6] > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > 1024,1024. > > 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > ntor=[0-6]' > > 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > such device > > > > Because /bin/pkg-config --static --libs libdpdk > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > -l:librte_common_cpt.a -l:librte_common_dpaax.a > > -l:librte_common_iavf.a -l:librte_common_octeontx.a > > -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > -l:librte_common_mlx5.a -l:librte_common_qat.a > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > > -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > > -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > > -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > > -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > > -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > > -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > > -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > > -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > > -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > > > > Note that: > > -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't > work.) > > I guess we dont hope that, It should be: > > -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > > > update pkg-config to pkgconf-1.4.2 > > /bin/pkg-config --static --libs libdpdk > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > -l:librte_common_dpaax.a -l:librte_common_iavf.a > > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > > -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > > > ovs works fine. > > Hmm. Thanks for the report. > > It'll be great if you can test this with new version of this patch: > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1- > git-send-email-ian.stokes@intel.com/ > > Bruce, David, it looks like not all versions of pkg-config could be used. > Do you know what is the minimal required version or how to protect us from > this kind of issues? > > Best regards, Ilya Maximets. This was just recently flagged to us and discussed on this email thread on the dpdk dev list. http://inbox.dpdk.org/dev/20201029091638.26646-1-getelson@nvidia.com/ At the end of the discussion the follow was noted in our docs as a known issue for DPDK. "pkg-config v0.27 supplied with RHEL-7 does not process correctly libdpdk.pc Libs.private section." If you have any suggestions as to how we could work around this issue, I'd be very keen to hear them, though unfortunately it does seem to be a bug in pkg-config that is outside of our control. Regards, /Bruce
On 12/15/20 10:43 AM, Richardson, Bruce wrote: > > >> -----Original Message----- >> From: Ilya Maximets <i.maximets@ovn.org> >> Sent: Tuesday, December 15, 2020 9:33 AM >> To: Tonghao Zhang <xiangxia.m.yue@gmail.com>; Pai G, Sunil >> <sunil.pai.g@intel.com> >> Cc: ovs dev <dev@openvswitch.org>; Richardson, Bruce >> <bruce.richardson@intel.com>; Ilya Maximets <i.maximets@ovn.org>; David >> Marchand <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> >> Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK >> meson build. >> >> On 12/15/20 5:02 AM, Tonghao Zhang wrote: >>> On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> >> wrote: >>>> >>>> Make based build is deprecated in DPDK. Meson based >>>> build to be used for future DPDK releases. >>>> >>>> This updates travis, configure script and documentation >>>> for using DPDK Meson with OVS. >>>> >>>> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs- >> copy/builds/723510063 >>>> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> >>>> --- >>>> v3->v4: >>>> - Fix checkpatch errors >>>> >>>> v2->v3: >>>> - Update Documentation for vhost-user >>>> >>>> v1->v2: >>>> - Update Documentation >>>> - Simplify the pkg-config parsing script >>>> - Rename and move the pkg-config parsing script to python dir >>>> - Update travis to: >>>> - install DPDK to cached dir >>>> - disable DPDK tests >>>> - removed fPIC flag for DPDK >>>> - removed cross compilation for aarch64 >>>> --- >>>> .travis.yml | 3 ++ >>>> .travis/linux-build.sh | 39 ++++++++++----- >>>> .travis/linux-prepare.sh | 1 + >>>> Documentation/intro/install/afxdp.rst | 2 +- >>>> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- >>>> Documentation/topics/dpdk/vhost-user.rst | 18 +------ >>>> acinclude.m4 | 44 +++++++++++------ >>>> python/automake.mk | 3 +- >>>> python/build/pkgcfg.py | 30 +++++++++++ >>>> 9 files changed, 149 insertions(+), 54 deletions(-) >>>> create mode 100644 python/build/pkgcfg.py >>>> >>>> diff --git a/.travis.yml b/.travis.yml >>>> index 3dd5d1d23..a8f9a4d79 100644 >>>> --- a/.travis.yml >>>> +++ b/.travis.yml >>>> @@ -27,6 +27,9 @@ addons: >>>> - selinux-policy-dev >>>> - libunbound-dev >>>> - libunwind-dev >>>> + - python3-setuptools >>>> + - python3-wheel >>>> + - ninja-build >>>> >>>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >>>> >>>> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >>>> index 817bf24aa..14ef833c9 100755 >>>> --- a/.travis/linux-build.sh >>>> +++ b/.travis/linux-build.sh >>>> @@ -85,17 +85,29 @@ function install_dpdk() >>>> { >>>> local DPDK_VER=$1 >>>> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" >>>> + local DPDK_OPTS="" >>>> + local DPDK_LIB="" >>>> >>>> if [ -z "$TRAVIS_ARCH" ] || >>>> [ "$TRAVIS_ARCH" == "amd64" ]; then >>>> - TARGET="x86_64-native-linuxapp-gcc" >>>> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu >>>> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then >>>> - TARGET="arm64-armv8a-linuxapp-gcc" >>>> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu >>>> else >>>> echo "Target is unknown" >>>> exit 1 >>>> fi >>>> >>>> + if [ "$DPDK_SHARED" ]; then >>>> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" >>>> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >>>> + else >>>> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" >>>> + fi >>>> + >>>> + # Export the following path for pkg-config to find the .pc file. >>>> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >>>> + >>>> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then >>>> # Avoid using cache for git tree build. >>>> rm -rf dpdk-dir >>>> @@ -108,7 +120,8 @@ function install_dpdk() >>>> if [ -f "${VERSION_FILE}" ]; then >>>> VER=$(cat ${VERSION_FILE}) >>>> if [ "${VER}" = "${DPDK_VER}" ]; then >>>> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk- >> dir/build" >>>> + # Update the library paths. >>>> + sudo ldconfig >>>> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk- >> dir" >>>> return >>>> fi >>>> @@ -122,16 +135,20 @@ function install_dpdk() >>>> pushd dpdk-dir >>>> fi >>>> >>>> - make config CC=gcc T=$TARGET >>>> + # Disable building DPDK unit tests. Not needed for OVS build or >> tests. >>>> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" >>>> >>>> - if [ "$DPDK_SHARED" ]; then >>>> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config >>>> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib >>>> - fi >>>> + # Install DPDK using prefix. >>>> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" >>>> + >>>> + CC=gcc meson $DPDK_OPTS build >>>> + ninja -C build >>>> + sudo ninja -C build install >>>> + >>>> + # Update the library paths. >>>> + sudo ldconfig >>>> >>>> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' >>>> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" >>>> - echo "Installed DPDK source in $(pwd)" >>>> + echo "Installed DPDK source" >>>> popd >>>> echo "${DPDK_VER}" > ${VERSION_FILE} >>>> } >>>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >>>> index 71eb347e8..1baa11641 100755 >>>> --- a/.travis/linux-prepare.sh >>>> +++ b/.travis/linux-prepare.sh >>>> @@ -22,6 +22,7 @@ cd .. >>>> >>>> pip3 install --disable-pip-version-check --user flake8 hacking >>>> pip3 install --user --upgrade docutils >>>> +pip3 install --user 'meson==0.47.1' >>>> >>>> if [ "$M32" ]; then >>>> # Installing 32-bit libraries. >>>> diff --git a/Documentation/intro/install/afxdp.rst >> b/Documentation/intro/install/afxdp.rst >>>> index 3c8f78825..327f2b3df 100644 >>>> --- a/Documentation/intro/install/afxdp.rst >>>> +++ b/Documentation/intro/install/afxdp.rst >>>> @@ -396,7 +396,7 @@ PVP using vhostuser device >>>> -------------------------- >>>> First, build OVS with DPDK and AFXDP:: >>>> >>>> - ./configure --enable-afxdp --with-dpdk=<dpdk path> >>>> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> >>>> make -j4 && make install >>>> >>>> Create a vhost-user port from OVS:: >>>> diff --git a/Documentation/intro/install/dpdk.rst >> b/Documentation/intro/install/dpdk.rst >>>> index 39544f835..cd7e51c75 100644 >>>> --- a/Documentation/intro/install/dpdk.rst >>>> +++ b/Documentation/intro/install/dpdk.rst >>>> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK >> requirements`_. >>>> .. _DPDK supported NIC: http://dpdk.org/doc/nics >>>> .. _DPDK requirements: >> http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html >>>> >>>> +.. _dpdk-install: >>>> + >>>> Installing >>>> ---------- >>>> >>>> @@ -76,10 +78,31 @@ Install DPDK >>>> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 >>>> $ cd $DPDK_DIR >>>> >>>> +#. Configure and install DPDK using Meson >>>> + >>>> + Meson is the preferred tool to build recent DPDK releases >>>> + as Make support is deprecated and will be removed from DPDK 20.11. >>>> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. >>>> + >>>> + Build and install the DPDK library:: >>>> + >>>> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >>>> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >>>> + $ meson $DPDK_TARGET >>>> + $ ninja -C $DPDK_TARGET >>>> + $ sudo ninja -C $DPDK_TARGET install >>>> + $ sudo ldconfig >>>> + >>>> + Detailed information can be found at `DPDK documentation`_. >>>> + >>>> #. (Optional) Configure DPDK as a shared library >>>> >>>> - DPDK can be built as either a static library or a shared library. >> By >>>> - default, it is configured for the former. If you wish to use the >> latter, set >>>> + When using Meson, DPDK is built both as static and shared library. >>>> + So no extra configuration is required in this case. >>>> + >>>> + In case of Make, DPDK can be built as either a static library or a >> shared >>>> + library. By default, it is configured for the former. If you wish >> to use >>>> + the latter, set >>>> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in >> ``$DPDK_DIR/config/common_base``. >>>> >>>> .. note:: >>>> @@ -87,7 +110,7 @@ Install DPDK >>>> Minor performance loss is expected when using OVS with a shared >> DPDK >>>> library compared to a static DPDK library. >>>> >>>> -#. Configure and install DPDK >>>> +#. Configure and install DPDK using Make >>>> >>>> Build and install the DPDK library:: >>>> >>>> @@ -97,12 +120,22 @@ Install DPDK >>>> >>>> #. (Optional) Export the DPDK shared library location >>>> >>>> - If DPDK was built as a shared library, export the path to this >> library for >>>> - use when building OVS:: >>>> + If DPDK was built as a shared library using Make, export the path >> to this >>>> + library for use when building OVS:: >>>> >>>> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp- >> gcc/lib >>>> >>>> + In case of Meson, exporting the path to library is not necessary if >>>> + the DPDK libraries are system installed. For libraries installed >> using >>>> + a prefix(assuming $DPDK_INSTALL in the below case), export the path >> to this >>>> + library and also update the $PKG_CONFIG_PATH for use before >> building OVS:: >>>> + >>>> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu >>>> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH >>>> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH >>>> + >>>> .. _DPDK sources: http://dpdk.org/rel >>>> +.. _DPDK documentation: >> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html >>>> >>>> Install OVS >>>> ~~~~~~~~~~~ >>>> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK >> library (``--with-dpdk``). >>>> >>>> #. Bootstrap, if required, as described in :ref:`general- >> bootstrapping` >>>> >>>> -#. Configure the package using the ``--with-dpdk`` flag:: >>>> +#. Configure the package using the ``--with-dpdk`` flag: >>>> + >>>> + Depending on the tool used to build DPDK and the type of >>>> + DPDK library to use, one can configure OVS as follows: >>>> + >>>> + When DPDK is built using Meson, and OVS must consume DPDK shared >> libraries >>>> + (also equivalent to leaving --with-dpdk option empty):: >>>> + >>>> + $ ./configure --with-dpdk=shared >>> Hi all >>> when I build ovs with --with-dpdk=shared, Compilation is fine, but we >>> I launch the ovs: >>> >>> 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a >>> >> 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen >> tor=[0-6] >>> -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 >>> --socket-limit 1024,1024. >>> 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) >>> 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes >>> 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device >>> "0000:82:00.0" >>> 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device >>> >> '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe >> ntor=[0-6]' >>> 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: >>> No such device >>> >>> In dpdk: >>> eal_plugins_init >>> rte_pci_scan >>> rte_bus_register >>> eal_option_device_parse -- no pci bus >>> >>> pkg-config is pkgconf-1.4.2 >>>> + When DPDK is built using Meson, and OVS must consume DPDK static >> libraries:: >>>> + >>>> + $ ./configure --with-dpdk=static >>> Only one issue, if we use the pkgconfig-0.27 >>> The ovs log: >>> 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a >>> >> 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen >> tor=[0-6] >>> -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit >>> 1024,1024. >>> 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) >>> 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes >>> 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device >>> "0000:82:00.0" >>> 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device >>> >> '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe >> ntor=[0-6]' >>> 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No >>> such device >>> >>> Because /bin/pkg-config --static --libs libdpdk >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive >>> -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 >>> -l:librte_common_cpt.a -l:librte_common_dpaax.a >>> -l:librte_common_iavf.a -l:librte_common_octeontx.a >>> -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a >>> -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a >>> -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a >>> -l:librte_common_mlx5.a -l:librte_common_qat.a >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a >>> -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a >>> -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a >>> -l:librte_net_virtio.a -l:librte_net_vmxnet3.a >>> -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a >>> -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a >>> -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a >>> -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a >>> -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a >>> -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a >>> -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a >>> -l:librte_power.a -l:librte_member.a -l:librte_lpm.a >>> -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a >>> -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a >>> -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a >>> -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a >>> -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a >>> -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf >>> -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib >>> -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched >>> -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump >>> -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni >>> -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev >>> -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev >>> -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer >>> -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev >>> -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring >>> -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' >>> >>> Note that: >>> -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't >> work.) >>> I guess we dont hope that, It should be: >>> -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive >>> >>> update pkg-config to pkgconf-1.4.2 >>> /bin/pkg-config --static --libs libdpdk >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive >>> -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a >>> -l:librte_common_dpaax.a -l:librte_common_iavf.a >>> -l:librte_common_octeontx.a -l:librte_common_octeontx2.a >>> -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a >>> -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a >>> -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a >>> -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a >>> -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a >>> -l:librte_net_virtio.a -l:librte_net_vmxnet3.a >>> -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a >>> -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a >>> -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a >>> -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a >>> -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a >>> -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a >>> -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a >>> -l:librte_power.a -l:librte_member.a -l:librte_lpm.a >>> -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a >>> -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a >>> -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a >>> -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a >>> -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a >>> -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs >>> -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter >>> -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal >>> -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' >>> >>> ovs works fine. >> >> Hmm. Thanks for the report. >> >> It'll be great if you can test this with new version of this patch: >> https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1- >> git-send-email-ian.stokes@intel.com/ >> >> Bruce, David, it looks like not all versions of pkg-config could be used. >> Do you know what is the minimal required version or how to protect us from >> this kind of issues? >> >> Best regards, Ilya Maximets. > > This was just recently flagged to us and discussed on this email thread on > the dpdk dev list. > > http://inbox.dpdk.org/dev/20201029091638.26646-1-getelson@nvidia.com/ > > At the end of the discussion the follow was noted in our docs as a known > issue for DPDK. > > "pkg-config v0.27 supplied with RHEL-7 does not process correctly libdpdk.pc Libs.private section." > > If you have any suggestions as to how we could work around this issue, I'd > be very keen to hear them, though unfortunately it does seem to be a bug > in pkg-config that is outside of our control. Thanks for the information. Ugh. It looks like we need to implement the check in our configure script and abort if pkg-config is bad to avoid weired runtime issues due to not linked libs. Probably, a grep, similar to what is in above dpdk patch, over the DPDK_vswitchd_LDFLAGS with some meaningfull error message should do the trick. Ian, what do you think? Best regards, Ilya Maximets.
> On 12/15/20 10:43 AM, Richardson, Bruce wrote: > > > > > >> -----Original Message----- > >> From: Ilya Maximets <i.maximets@ovn.org> > >> Sent: Tuesday, December 15, 2020 9:33 AM > >> To: Tonghao Zhang <xiangxia.m.yue@gmail.com>; Pai G, Sunil > >> <sunil.pai.g@intel.com> > >> Cc: ovs dev <dev@openvswitch.org>; Richardson, Bruce > >> <bruce.richardson@intel.com>; Ilya Maximets <i.maximets@ovn.org>; David > >> Marchand <david.marchand@redhat.com>; Stokes, Ian > <ian.stokes@intel.com> > >> Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > >> meson build. > >> > >> On 12/15/20 5:02 AM, Tonghao Zhang wrote: > >>> On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> > >> wrote: > >>>> > >>>> Make based build is deprecated in DPDK. Meson based > >>>> build to be used for future DPDK releases. > >>>> > >>>> This updates travis, configure script and documentation > >>>> for using DPDK Meson with OVS. > >>>> > >>>> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs- > >> copy/builds/723510063 > >>>> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > >>>> --- > >>>> v3->v4: > >>>> - Fix checkpatch errors > >>>> > >>>> v2->v3: > >>>> - Update Documentation for vhost-user > >>>> > >>>> v1->v2: > >>>> - Update Documentation > >>>> - Simplify the pkg-config parsing script > >>>> - Rename and move the pkg-config parsing script to python dir > >>>> - Update travis to: > >>>> - install DPDK to cached dir > >>>> - disable DPDK tests > >>>> - removed fPIC flag for DPDK > >>>> - removed cross compilation for aarch64 > >>>> --- > >>>> .travis.yml | 3 ++ > >>>> .travis/linux-build.sh | 39 ++++++++++----- > >>>> .travis/linux-prepare.sh | 1 + > >>>> Documentation/intro/install/afxdp.rst | 2 +- > >>>> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > >>>> Documentation/topics/dpdk/vhost-user.rst | 18 +------ > >>>> acinclude.m4 | 44 +++++++++++------ > >>>> python/automake.mk | 3 +- > >>>> python/build/pkgcfg.py | 30 +++++++++++ > >>>> 9 files changed, 149 insertions(+), 54 deletions(-) > >>>> create mode 100644 python/build/pkgcfg.py > >>>> > >>>> diff --git a/.travis.yml b/.travis.yml > >>>> index 3dd5d1d23..a8f9a4d79 100644 > >>>> --- a/.travis.yml > >>>> +++ b/.travis.yml > >>>> @@ -27,6 +27,9 @@ addons: > >>>> - selinux-policy-dev > >>>> - libunbound-dev > >>>> - libunwind-dev > >>>> + - python3-setuptools > >>>> + - python3-wheel > >>>> + - ninja-build > >>>> > >>>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > >>>> > >>>> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > >>>> index 817bf24aa..14ef833c9 100755 > >>>> --- a/.travis/linux-build.sh > >>>> +++ b/.travis/linux-build.sh > >>>> @@ -85,17 +85,29 @@ function install_dpdk() > >>>> { > >>>> local DPDK_VER=$1 > >>>> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > >>>> + local DPDK_OPTS="" > >>>> + local DPDK_LIB="" > >>>> > >>>> if [ -z "$TRAVIS_ARCH" ] || > >>>> [ "$TRAVIS_ARCH" == "amd64" ]; then > >>>> - TARGET="x86_64-native-linuxapp-gcc" > >>>> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > >>>> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > >>>> - TARGET="arm64-armv8a-linuxapp-gcc" > >>>> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > >>>> else > >>>> echo "Target is unknown" > >>>> exit 1 > >>>> fi > >>>> > >>>> + if [ "$DPDK_SHARED" ]; then > >>>> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > >>>> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >>>> + else > >>>> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > >>>> + fi > >>>> + > >>>> + # Export the following path for pkg-config to find the .pc file. > >>>> + export > PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >>>> + > >>>> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > >>>> # Avoid using cache for git tree build. > >>>> rm -rf dpdk-dir > >>>> @@ -108,7 +120,8 @@ function install_dpdk() > >>>> if [ -f "${VERSION_FILE}" ]; then > >>>> VER=$(cat ${VERSION_FILE}) > >>>> if [ "${VER}" = "${DPDK_VER}" ]; then > >>>> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk- > >> dir/build" > >>>> + # Update the library paths. > >>>> + sudo ldconfig > >>>> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk- > >> dir" > >>>> return > >>>> fi > >>>> @@ -122,16 +135,20 @@ function install_dpdk() > >>>> pushd dpdk-dir > >>>> fi > >>>> > >>>> - make config CC=gcc T=$TARGET > >>>> + # Disable building DPDK unit tests. Not needed for OVS build or > >> tests. > >>>> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > >>>> > >>>> - if [ "$DPDK_SHARED" ]; then > >>>> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > >>>> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > >>>> - fi > >>>> + # Install DPDK using prefix. > >>>> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > >>>> + > >>>> + CC=gcc meson $DPDK_OPTS build > >>>> + ninja -C build > >>>> + sudo ninja -C build install > >>>> + > >>>> + # Update the library paths. > >>>> + sudo ldconfig > >>>> > >>>> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > >>>> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > >>>> - echo "Installed DPDK source in $(pwd)" > >>>> + echo "Installed DPDK source" > >>>> popd > >>>> echo "${DPDK_VER}" > ${VERSION_FILE} > >>>> } > >>>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > >>>> index 71eb347e8..1baa11641 100755 > >>>> --- a/.travis/linux-prepare.sh > >>>> +++ b/.travis/linux-prepare.sh > >>>> @@ -22,6 +22,7 @@ cd .. > >>>> > >>>> pip3 install --disable-pip-version-check --user flake8 hacking > >>>> pip3 install --user --upgrade docutils > >>>> +pip3 install --user 'meson==0.47.1' > >>>> > >>>> if [ "$M32" ]; then > >>>> # Installing 32-bit libraries. > >>>> diff --git a/Documentation/intro/install/afxdp.rst > >> b/Documentation/intro/install/afxdp.rst > >>>> index 3c8f78825..327f2b3df 100644 > >>>> --- a/Documentation/intro/install/afxdp.rst > >>>> +++ b/Documentation/intro/install/afxdp.rst > >>>> @@ -396,7 +396,7 @@ PVP using vhostuser device > >>>> -------------------------- > >>>> First, build OVS with DPDK and AFXDP:: > >>>> > >>>> - ./configure --enable-afxdp --with-dpdk=<dpdk path> > >>>> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > >>>> make -j4 && make install > >>>> > >>>> Create a vhost-user port from OVS:: > >>>> diff --git a/Documentation/intro/install/dpdk.rst > >> b/Documentation/intro/install/dpdk.rst > >>>> index 39544f835..cd7e51c75 100644 > >>>> --- a/Documentation/intro/install/dpdk.rst > >>>> +++ b/Documentation/intro/install/dpdk.rst > >>>> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > >> requirements`_. > >>>> .. _DPDK supported NIC: http://dpdk.org/doc/nics > >>>> .. _DPDK requirements: > >> http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > >>>> > >>>> +.. _dpdk-install: > >>>> + > >>>> Installing > >>>> ---------- > >>>> > >>>> @@ -76,10 +78,31 @@ Install DPDK > >>>> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > >>>> $ cd $DPDK_DIR > >>>> > >>>> +#. Configure and install DPDK using Meson > >>>> + > >>>> + Meson is the preferred tool to build recent DPDK releases > >>>> + as Make support is deprecated and will be removed from DPDK 20.11. > >>>> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > >>>> + > >>>> + Build and install the DPDK library:: > >>>> + > >>>> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > >>>> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > >>>> + $ meson $DPDK_TARGET > >>>> + $ ninja -C $DPDK_TARGET > >>>> + $ sudo ninja -C $DPDK_TARGET install > >>>> + $ sudo ldconfig > >>>> + > >>>> + Detailed information can be found at `DPDK documentation`_. > >>>> + > >>>> #. (Optional) Configure DPDK as a shared library > >>>> > >>>> - DPDK can be built as either a static library or a shared library. > >> By > >>>> - default, it is configured for the former. If you wish to use the > >> latter, set > >>>> + When using Meson, DPDK is built both as static and shared library. > >>>> + So no extra configuration is required in this case. > >>>> + > >>>> + In case of Make, DPDK can be built as either a static library or a > >> shared > >>>> + library. By default, it is configured for the former. If you wish > >> to use > >>>> + the latter, set > >>>> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > >> ``$DPDK_DIR/config/common_base``. > >>>> > >>>> .. note:: > >>>> @@ -87,7 +110,7 @@ Install DPDK > >>>> Minor performance loss is expected when using OVS with a shared > >> DPDK > >>>> library compared to a static DPDK library. > >>>> > >>>> -#. Configure and install DPDK > >>>> +#. Configure and install DPDK using Make > >>>> > >>>> Build and install the DPDK library:: > >>>> > >>>> @@ -97,12 +120,22 @@ Install DPDK > >>>> > >>>> #. (Optional) Export the DPDK shared library location > >>>> > >>>> - If DPDK was built as a shared library, export the path to this > >> library for > >>>> - use when building OVS:: > >>>> + If DPDK was built as a shared library using Make, export the path > >> to this > >>>> + library for use when building OVS:: > >>>> > >>>> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp- > >> gcc/lib > >>>> > >>>> + In case of Meson, exporting the path to library is not necessary if > >>>> + the DPDK libraries are system installed. For libraries installed > >> using > >>>> + a prefix(assuming $DPDK_INSTALL in the below case), export the path > >> to this > >>>> + library and also update the $PKG_CONFIG_PATH for use before > >> building OVS:: > >>>> + > >>>> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > >>>> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >>>> + $ export > PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >>>> + > >>>> .. _DPDK sources: http://dpdk.org/rel > >>>> +.. _DPDK documentation: > >> https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > >>>> > >>>> Install OVS > >>>> ~~~~~~~~~~~ > >>>> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK > >> library (``--with-dpdk``). > >>>> > >>>> #. Bootstrap, if required, as described in :ref:`general- > >> bootstrapping` > >>>> > >>>> -#. Configure the package using the ``--with-dpdk`` flag:: > >>>> +#. Configure the package using the ``--with-dpdk`` flag: > >>>> + > >>>> + Depending on the tool used to build DPDK and the type of > >>>> + DPDK library to use, one can configure OVS as follows: > >>>> + > >>>> + When DPDK is built using Meson, and OVS must consume DPDK shared > >> libraries > >>>> + (also equivalent to leaving --with-dpdk option empty):: > >>>> + > >>>> + $ ./configure --with-dpdk=shared > >>> Hi all > >>> when I build ovs with --with-dpdk=shared, Compilation is fine, but we > >>> I launch the ovs: > >>> > >>> 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd - > a > >>> > >> > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,repre > sen > >> tor=[0-6] > >>> -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > >>> --socket-limit 1024,1024. > >>> 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > >>> 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > nodes > >>> 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > >>> "0000:82:00.0" > >>> 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse > device > >>> > >> > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,repr > ese > >> ntor=[0-6]' > >>> 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize > DPDK: > >>> No such device > >>> > >>> In dpdk: > >>> eal_plugins_init > >>> rte_pci_scan > >>> rte_bus_register > >>> eal_option_device_parse -- no pci bus > >>> > >>> pkg-config is pkgconf-1.4.2 > >>>> + When DPDK is built using Meson, and OVS must consume DPDK static > >> libraries:: > >>>> + > >>>> + $ ./configure --with-dpdk=static > >>> Only one issue, if we use the pkgconfig-0.27 > >>> The ovs log: > >>> 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > >>> > >> > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,repre > sen > >> tor=[0-6] > >>> -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > >>> 1024,1024. > >>> 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > >>> 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > nodes > >>> 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > >>> "0000:82:00.0" > >>> 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > >>> > >> > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,repr > ese > >> ntor=[0-6]' > >>> 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: > No > >>> such device > >>> > >>> Because /bin/pkg-config --static --libs libdpdk > >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > >>> -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > >>> -l:librte_common_cpt.a -l:librte_common_dpaax.a > >>> -l:librte_common_iavf.a -l:librte_common_octeontx.a > >>> -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > >>> -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > >>> -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > >>> -l:librte_common_mlx5.a -l:librte_common_qat.a > >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a > >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > >>> -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > >>> -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > >>> -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > >>> -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > >>> -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > >>> -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > >>> -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > >>> -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > >>> -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > >>> -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > >>> -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > >>> -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > >>> -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > >>> -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > >>> -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > >>> -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > >>> -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > >>> -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > >>> -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > >>> -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > >>> -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > >>> -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > >>> -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > >>> -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > >>> -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > >>> -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > >>> -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > >>> > >>> Note that: > >>> -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't > >> work.) > >>> I guess we dont hope that, It should be: > >>> -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > >>> > >>> update pkg-config to pkgconf-1.4.2 > >>> /bin/pkg-config --static --libs libdpdk > >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive > >>> -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > >>> -l:librte_common_dpaax.a -l:librte_common_iavf.a > >>> -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > >>> -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > >>> -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > >>> -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a > >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > >>> -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > >>> -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > >>> -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > >>> -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > >>> -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > >>> -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > >>> -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > >>> -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > >>> -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > >>> -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > >>> -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > >>> -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > >>> -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > >>> -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > >>> -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > >>> -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > >>> -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > >>> -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > >>> -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > >>> -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > >>> > >>> ovs works fine. > >> > >> Hmm. Thanks for the report. > >> > >> It'll be great if you can test this with new version of this patch: > >> https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > 30877-1- > >> git-send-email-ian.stokes@intel.com/ > >> > >> Bruce, David, it looks like not all versions of pkg-config could be used. > >> Do you know what is the minimal required version or how to protect us from > >> this kind of issues? > >> > >> Best regards, Ilya Maximets. > > > > This was just recently flagged to us and discussed on this email thread on > > the dpdk dev list. > > > > http://inbox.dpdk.org/dev/20201029091638.26646-1-getelson@nvidia.com/ > > > > At the end of the discussion the follow was noted in our docs as a known > > issue for DPDK. > > > > "pkg-config v0.27 supplied with RHEL-7 does not process correctly libdpdk.pc > Libs.private section." > > > > If you have any suggestions as to how we could work around this issue, I'd > > be very keen to hear them, though unfortunately it does seem to be a bug > > in pkg-config that is outside of our control. > > Thanks for the information. Ugh. > > It looks like we need to implement the check in our configure script and abort > if pkg-config is bad to avoid weired runtime issues due to not linked libs. > > Probably, a grep, similar to what is in above dpdk patch, over the > DPDK_vswitchd_LDFLAGS with some meaningfull error message should do the > trick. > > Ian, what do you think? Sounds good, Let myself and Sunil take a look at this and send a new v4 with the check. Regards Ian > > Best regards, Ilya Maximets.
Hi, <snipped> > > >>>> + When DPDK is built using Meson, and OVS must consume DPDK > > >>>> + shared > > >> libraries > > >>>> + (also equivalent to leaving --with-dpdk option empty):: > > >>>> + > > >>>> + $ ./configure --with-dpdk=shared > > >>> Hi all > > >>> when I build ovs with --with-dpdk=shared, Compilation is fine, but > > >>> we I launch the ovs: > > >>> > > >>> 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs- > vswitchd > > >>> - > > a > > >>> > > >> > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > epr > > e > > sen > > >> tor=[0-6] > > >>> -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > >>> --socket-limit 1024,1024. > > >>> 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 > > >>> lcore(s) > > >>> 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 > NUMA > > nodes > > >>> 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse > > >>> device "0000:82:00.0" > > >>> 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse > > device > > >>> > > >> > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > ep > > r > > ese > > >> ntor=[0-6]' > > >>> 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize > > DPDK: > > >>> No such device > > >>> > > >>> In dpdk: > > >>> eal_plugins_init > > >>> rte_pci_scan > > >>> rte_bus_register > > >>> eal_option_device_parse -- no pci bus > > >>> > > >>> pkg-config is pkgconf-1.4.2 > > >>>> + When DPDK is built using Meson, and OVS must consume DPDK > > >>>> + static > > >> libraries:: > > >>>> + > > >>>> + $ ./configure --with-dpdk=static > > >>> Only one issue, if we use the pkgconfig-0.27 The ovs log: > > >>> 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd > -a > > >>> > > >> > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > epr > > e > > sen > > >> tor=[0-6] > > >>> -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > >>> --socket-limit 1024,1024. > > >>> 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > >>> 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > nodes > > >>> 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse > > >>> device "0000:82:00.0" > > >>> 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse > > >>> device > > >>> > > >> > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > ep > > r > > ese > > >> ntor=[0-6]' > > >>> 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize > DPDK: > > No > > >>> such device > > >>> > > >>> Because /bin/pkg-config --static --libs libdpdk > > >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > >>> -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > >>> -l:librte_common_cpt.a -l:librte_common_dpaax.a > > >>> -l:librte_common_iavf.a -l:librte_common_octeontx.a > > >>> -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > >>> -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > >>> -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > >>> -l:librte_common_mlx5.a -l:librte_common_qat.a > > >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > >>> -l:librte_net_enic.a -l:librte_net_failsafe.a > > >>> -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a > > >>> -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a > > >>> -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a > > >>> -l:librte_net_liquidio.a -l:librte_net_memif.a > > >>> -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a > > >>> -l:librte_net_null.a -l:librte_net_octeontx.a > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > > >>> -l:librte_baseband_fpga_5gnr_fec.a > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a > > >>> -l:librte_distributor.a -l:librte_cryptodev.a > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -lmlx5 > > >>> -libverbs -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry - > lrte_kvargs -lm -ldl -lnuma -lpcap ' > > >>> > > >>> Note that: > > >>> -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk > > >>> doesn't > > >> work.) > > >>> I guess we dont hope that, It should be: > > >>> -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > >>> > > >>> update pkg-config to pkgconf-1.4.2 /bin/pkg-config --static --libs > > >>> libdpdk pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > >>> -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > >>> -l:librte_common_dpaax.a -l:librte_common_iavf.a > > >>> -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > >>> -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a > > >>> -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a > > >>> -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a > > >>> -l:librte_common_qat.a -l:librte_mempool_bucket.a > > >>> -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a > > >>> -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a > > >>> -l:librte_mempool_ring.a -l:librte_mempool_stack.a > > >>> -l:librte_net_af_packet.a -l:librte_net_ark.a > > >>> -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a > > >>> -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a > > >>> -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a > > >>> -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a > > >>> -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > > >>> -l:librte_baseband_fpga_5gnr_fec.a > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a > > >>> -l:librte_distributor.a -l:librte_cryptodev.a > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a > > >>> -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs -Wl,--as-needed > > >>> -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry - > lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > >>> > > >>> ovs works fine. > > >> > > >> Hmm. Thanks for the report. > > >> > > >> It'll be great if you can test this with new version of this patch: > > >> https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > 30877-1- > > >> git-send-email-ian.stokes@intel.com/ > > >> > > >> Bruce, David, it looks like not all versions of pkg-config could be used. > > >> Do you know what is the minimal required version or how to protect > > >> us from this kind of issues? > > >> > > >> Best regards, Ilya Maximets. > > > > > > This was just recently flagged to us and discussed on this email > > > thread on the dpdk dev list. > > > > > > http://inbox.dpdk.org/dev/20201029091638.26646-1- > getelson@nvidia.com > > > / > > > > > > At the end of the discussion the follow was noted in our docs as a > > > known issue for DPDK. > > > > > > "pkg-config v0.27 supplied with RHEL-7 does not process correctly > > > libdpdk.pc > > Libs.private section." > > > > > > If you have any suggestions as to how we could work around this > > > issue, I'd be very keen to hear them, though unfortunately it does > > > seem to be a bug in pkg-config that is outside of our control. > > > > Thanks for the information. Ugh. > > > > It looks like we need to implement the check in our configure script > > and abort if pkg-config is bad to avoid weired runtime issues due to not > linked libs. > > > > Probably, a grep, similar to what is in above dpdk patch, over the > > DPDK_vswitchd_LDFLAGS with some meaningfull error message should do > > the trick. > > > > Ian, what do you think? > > Sounds good, Let myself and Sunil take a look at this and send a new v4 with > the check. Sure , I think this could be added. But , if there is a check already in place in DPDK, would we need one in OVS as well ? > > Regards > Ian > > > > Best regards, Ilya Maximets.
> -----Original Message----- > From: Pai G, Sunil <sunil.pai.g@intel.com> > Sent: Tuesday, December 15, 2020 12:01 PM > To: Stokes, Ian <ian.stokes@intel.com>; Ilya Maximets > <i.maximets@ovn.org>; Richardson, Bruce <bruce.richardson@intel.com>; > Tonghao Zhang <xiangxia.m.yue@gmail.com> > Cc: ovs dev <dev@openvswitch.org>; David Marchand > <david.marchand@redhat.com> > Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > Hi, > > <snipped> > > > > >>>> + When DPDK is built using Meson, and OVS must consume DPDK > > > >>>> + shared > > > >> libraries > > > >>>> + (also equivalent to leaving --with-dpdk option empty):: > > > >>>> + > > > >>>> + $ ./configure --with-dpdk=shared > > > >>> Hi all > > > >>> when I build ovs with --with-dpdk=shared, Compilation is fine, but > > > >>> we I launch the ovs: > > > >>> > > > >>> 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs- > > vswitchd > > > >>> - > > > a > > > >>> > > > >> > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > epr > > > e > > > sen > > > >> tor=[0-6] > > > >>> -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > > >>> --socket-limit 1024,1024. > > > >>> 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 > > > >>> lcore(s) > > > >>> 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 > > NUMA > > > nodes > > > >>> 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse > > > >>> device "0000:82:00.0" > > > >>> 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse > > > device > > > >>> > > > >> > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > ep > > > r > > > ese > > > >> ntor=[0-6]' > > > >>> 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize > > > DPDK: > > > >>> No such device > > > >>> > > > >>> In dpdk: > > > >>> eal_plugins_init > > > >>> rte_pci_scan > > > >>> rte_bus_register > > > >>> eal_option_device_parse -- no pci bus > > > >>> > > > >>> pkg-config is pkgconf-1.4.2 > > > >>>> + When DPDK is built using Meson, and OVS must consume DPDK > > > >>>> + static > > > >> libraries:: > > > >>>> + > > > >>>> + $ ./configure --with-dpdk=static > > > >>> Only one issue, if we use the pkgconfig-0.27 The ovs log: > > > >>> 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd > > -a > > > >>> > > > >> > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > epr > > > e > > > sen > > > >> tor=[0-6] > > > >>> -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > > >>> --socket-limit 1024,1024. > > > >>> 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > >>> 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > > nodes > > > >>> 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse > > > >>> device "0000:82:00.0" > > > >>> 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse > > > >>> device > > > >>> > > > >> > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > ep > > > r > > > ese > > > >> ntor=[0-6]' > > > >>> 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize > > DPDK: > > > No > > > >>> such device > > > >>> > > > >>> Because /bin/pkg-config --static --libs libdpdk > > > >>> pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > > >>> -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > > >>> -l:librte_common_cpt.a -l:librte_common_dpaax.a > > > >>> -l:librte_common_iavf.a -l:librte_common_octeontx.a > > > >>> -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > > >>> -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > > >>> -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > > >>> -l:librte_common_mlx5.a -l:librte_common_qat.a > > > >>> -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > > >>> -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > > >>> -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > > >>> -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > > >>> -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > > >>> -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > > >>> -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > > >>> -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > > >>> -l:librte_net_enic.a -l:librte_net_failsafe.a > > > >>> -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a > > > >>> -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a > > > >>> -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a > > > >>> -l:librte_net_liquidio.a -l:librte_net_memif.a > > > >>> -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a > > > >>> -l:librte_net_null.a -l:librte_net_octeontx.a > > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a > > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a > > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a > > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a > > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a > > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > > > >>> -l:librte_baseband_fpga_5gnr_fec.a > > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a > > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a > > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a > > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a > > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a > > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a > > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a > > > >>> -l:librte_distributor.a -l:librte_cryptodev.a > > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a > > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a > > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a > > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -lmlx5 > > > >>> -libverbs -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter - > lrte_net > > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry > - > > lrte_kvargs -lm -ldl -lnuma -lpcap ' > > > >>> > > > >>> Note that: > > > >>> -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk > > > >>> doesn't > > > >> work.) > > > >>> I guess we dont hope that, It should be: > > > >>> -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > > >>> > > > >>> update pkg-config to pkgconf-1.4.2 /bin/pkg-config --static --libs > > > >>> libdpdk pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > > >>> -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > > >>> -l:librte_common_dpaax.a -l:librte_common_iavf.a > > > >>> -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > > >>> -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a > > > >>> -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a > > > >>> -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a > > > >>> -l:librte_common_qat.a -l:librte_mempool_bucket.a > > > >>> -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a > > > >>> -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a > > > >>> -l:librte_mempool_ring.a -l:librte_mempool_stack.a > > > >>> -l:librte_net_af_packet.a -l:librte_net_ark.a > > > >>> -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a > > > >>> -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a > > > >>> -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a > > > >>> -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a > > > >>> -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > > >>> -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > > >>> -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > > >>> -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > > >>> -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > > >>> -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > > >>> -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > >>> -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > >>> -l:librte_net_softnic.a -l:librte_net_thunderx.a > > > >>> -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > > > >>> -l:librte_net_vhost.a -l:librte_net_virtio.a > > > >>> -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a > > > >>> -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > > > >>> -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > >>> -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > >>> -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > >>> -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a > > > >>> -l:librte_vdpa_mlx5.a -l:librte_baseband_null.a > > > >>> -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > > > >>> -l:librte_baseband_fpga_5gnr_fec.a > > > >>> -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > >>> -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > >>> -l:librte_table.a -l:librte_port.a -l:librte_fib.a > > > >>> -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a > > > >>> -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a > > > >>> -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > > > >>> -l:librte_pdump.a -l:librte_power.a -l:librte_member.a > > > >>> -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a > > > >>> -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a > > > >>> -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a > > > >>> -l:librte_distributor.a -l:librte_cryptodev.a > > > >>> -l:librte_compressdev.a -l:librte_cfgfile.a > > > >>> -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > >>> -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > >>> -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > >>> -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a > > > >>> -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a > > > >>> -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a > > > >>> -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs -Wl,--as-needed > > > >>> -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > > >>> -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > > >>> -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > > >>> -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > > >>> -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > > >>> -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > > >>> -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > > >>> -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > > >>> -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter - > lrte_net > > -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry > - > > lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > > >>> > > > >>> ovs works fine. > > > >> > > > >> Hmm. Thanks for the report. > > > >> > > > >> It'll be great if you can test this with new version of this patch: > > > >> https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > 30877-1- > > > >> git-send-email-ian.stokes@intel.com/ > > > >> > > > >> Bruce, David, it looks like not all versions of pkg-config could be > used. > > > >> Do you know what is the minimal required version or how to protect > > > >> us from this kind of issues? > > > >> > > > >> Best regards, Ilya Maximets. > > > > > > > > This was just recently flagged to us and discussed on this email > > > > thread on the dpdk dev list. > > > > > > > > http://inbox.dpdk.org/dev/20201029091638.26646-1- > > getelson@nvidia.com > > > > / > > > > > > > > At the end of the discussion the follow was noted in our docs as a > > > > known issue for DPDK. > > > > > > > > "pkg-config v0.27 supplied with RHEL-7 does not process correctly > > > > libdpdk.pc > > > Libs.private section." > > > > > > > > If you have any suggestions as to how we could work around this > > > > issue, I'd be very keen to hear them, though unfortunately it does > > > > seem to be a bug in pkg-config that is outside of our control. > > > > > > Thanks for the information. Ugh. > > > > > > It looks like we need to implement the check in our configure script > > > and abort if pkg-config is bad to avoid weired runtime issues due to > not > > linked libs. > > > > > > Probably, a grep, similar to what is in above dpdk patch, over the > > > DPDK_vswitchd_LDFLAGS with some meaningfull error message should do > > > the trick. > > > > > > Ian, what do you think? > > > > Sounds good, Let myself and Sunil take a look at this and send a new v4 > with > > the check. > > Sure , I think this could be added. > But , if there is a check already in place in DPDK, would we need one in > OVS as well ? > There is not a check in place in DPDK, because while we can check the pkg-config version on the build machine, it's the version on the end-user machine (the OVS build machine in this case) that matters, and we can't check that as part of a DPDK build. /Bruce
<snipped> > > > > >> Hmm. Thanks for the report. > > > > >> > > > > >> It'll be great if you can test this with new version of this patch: > > > > >> https://patchwork.ozlabs.org/project/openvswitch/patch/16079778 > > > > >> 25- > > > > 30877-1- > > > > >> git-send-email-ian.stokes@intel.com/ > > > > >> > > > > >> Bruce, David, it looks like not all versions of pkg-config > > > > >> could be > > used. > > > > >> Do you know what is the minimal required version or how to > > > > >> protect us from this kind of issues? > > > > >> > > > > >> Best regards, Ilya Maximets. > > > > > > > > > > This was just recently flagged to us and discussed on this email > > > > > thread on the dpdk dev list. > > > > > > > > > > http://inbox.dpdk.org/dev/20201029091638.26646-1- > > > getelson@nvidia.com > > > > > / > > > > > > > > > > At the end of the discussion the follow was noted in our docs as > > > > > a known issue for DPDK. > > > > > > > > > > "pkg-config v0.27 supplied with RHEL-7 does not process > > > > > correctly libdpdk.pc > > > > Libs.private section." > > > > > > > > > > If you have any suggestions as to how we could work around this > > > > > issue, I'd be very keen to hear them, though unfortunately it > > > > > does seem to be a bug in pkg-config that is outside of our control. > > > > > > > > Thanks for the information. Ugh. > > > > > > > > It looks like we need to implement the check in our configure > > > > script and abort if pkg-config is bad to avoid weired runtime > > > > issues due to > > not > > > linked libs. > > > > > > > > Probably, a grep, similar to what is in above dpdk patch, over the > > > > DPDK_vswitchd_LDFLAGS with some meaningfull error message should > > > > do the trick. > > > > > > > > Ian, what do you think? > > > > > > Sounds good, Let myself and Sunil take a look at this and send a new > > > v4 > > with > > > the check. > > > > Sure , I think this could be added. > > But , if there is a check already in place in DPDK, would we need one > > in OVS as well ? > > > > There is not a check in place in DPDK, because while we can check the pkg- > config version on the build machine, it's the version on the end-user machine > (the OVS build machine in this case) that matters, and we can't check that as > part of a DPDK build. > Agreed, Will add a check like DPDK in the configure script. > /Bruce
On Tue, Dec 15, 2020 at 5:32 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > On 12/15/20 5:02 AM, Tonghao Zhang wrote: > > On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> wrote: > >> > >> Make based build is deprecated in DPDK. Meson based > >> build to be used for future DPDK releases. > >> > >> This updates travis, configure script and documentation > >> for using DPDK Meson with OVS. > >> > >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 > >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > >> --- > >> v3->v4: > >> - Fix checkpatch errors > >> > >> v2->v3: > >> - Update Documentation for vhost-user > >> > >> v1->v2: > >> - Update Documentation > >> - Simplify the pkg-config parsing script > >> - Rename and move the pkg-config parsing script to python dir > >> - Update travis to: > >> - install DPDK to cached dir > >> - disable DPDK tests > >> - removed fPIC flag for DPDK > >> - removed cross compilation for aarch64 > >> --- > >> .travis.yml | 3 ++ > >> .travis/linux-build.sh | 39 ++++++++++----- > >> .travis/linux-prepare.sh | 1 + > >> Documentation/intro/install/afxdp.rst | 2 +- > >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- > >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ > >> acinclude.m4 | 44 +++++++++++------ > >> python/automake.mk | 3 +- > >> python/build/pkgcfg.py | 30 +++++++++++ > >> 9 files changed, 149 insertions(+), 54 deletions(-) > >> create mode 100644 python/build/pkgcfg.py > >> > >> diff --git a/.travis.yml b/.travis.yml > >> index 3dd5d1d23..a8f9a4d79 100644 > >> --- a/.travis.yml > >> +++ b/.travis.yml > >> @@ -27,6 +27,9 @@ addons: > >> - selinux-policy-dev > >> - libunbound-dev > >> - libunwind-dev > >> + - python3-setuptools > >> + - python3-wheel > >> + - ninja-build > >> > >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > >> > >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > >> index 817bf24aa..14ef833c9 100755 > >> --- a/.travis/linux-build.sh > >> +++ b/.travis/linux-build.sh > >> @@ -85,17 +85,29 @@ function install_dpdk() > >> { > >> local DPDK_VER=$1 > >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > >> + local DPDK_OPTS="" > >> + local DPDK_LIB="" > >> > >> if [ -z "$TRAVIS_ARCH" ] || > >> [ "$TRAVIS_ARCH" == "amd64" ]; then > >> - TARGET="x86_64-native-linuxapp-gcc" > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > >> - TARGET="arm64-armv8a-linuxapp-gcc" > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > >> else > >> echo "Target is unknown" > >> exit 1 > >> fi > >> > >> + if [ "$DPDK_SHARED" ]; then > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >> + else > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > >> + fi > >> + > >> + # Export the following path for pkg-config to find the .pc file. > >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >> + > >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > >> # Avoid using cache for git tree build. > >> rm -rf dpdk-dir > >> @@ -108,7 +120,8 @@ function install_dpdk() > >> if [ -f "${VERSION_FILE}" ]; then > >> VER=$(cat ${VERSION_FILE}) > >> if [ "${VER}" = "${DPDK_VER}" ]; then > >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" > >> + # Update the library paths. > >> + sudo ldconfig > >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" > >> return > >> fi > >> @@ -122,16 +135,20 @@ function install_dpdk() > >> pushd dpdk-dir > >> fi > >> > >> - make config CC=gcc T=$TARGET > >> + # Disable building DPDK unit tests. Not needed for OVS build or tests. > >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > >> > >> - if [ "$DPDK_SHARED" ]; then > >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config > >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > >> - fi > >> + # Install DPDK using prefix. > >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > >> + > >> + CC=gcc meson $DPDK_OPTS build > >> + ninja -C build > >> + sudo ninja -C build install > >> + > >> + # Update the library paths. > >> + sudo ldconfig > >> > >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > >> - echo "Installed DPDK source in $(pwd)" > >> + echo "Installed DPDK source" > >> popd > >> echo "${DPDK_VER}" > ${VERSION_FILE} > >> } > >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > >> index 71eb347e8..1baa11641 100755 > >> --- a/.travis/linux-prepare.sh > >> +++ b/.travis/linux-prepare.sh > >> @@ -22,6 +22,7 @@ cd .. > >> > >> pip3 install --disable-pip-version-check --user flake8 hacking > >> pip3 install --user --upgrade docutils > >> +pip3 install --user 'meson==0.47.1' > >> > >> if [ "$M32" ]; then > >> # Installing 32-bit libraries. > >> diff --git a/Documentation/intro/install/afxdp.rst b/Documentation/intro/install/afxdp.rst > >> index 3c8f78825..327f2b3df 100644 > >> --- a/Documentation/intro/install/afxdp.rst > >> +++ b/Documentation/intro/install/afxdp.rst > >> @@ -396,7 +396,7 @@ PVP using vhostuser device > >> -------------------------- > >> First, build OVS with DPDK and AFXDP:: > >> > >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> > >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > >> make -j4 && make install > >> > >> Create a vhost-user port from OVS:: > >> diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst > >> index 39544f835..cd7e51c75 100644 > >> --- a/Documentation/intro/install/dpdk.rst > >> +++ b/Documentation/intro/install/dpdk.rst > >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK requirements`_. > >> .. _DPDK supported NIC: http://dpdk.org/doc/nics > >> .. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > >> > >> +.. _dpdk-install: > >> + > >> Installing > >> ---------- > >> > >> @@ -76,10 +78,31 @@ Install DPDK > >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > >> $ cd $DPDK_DIR > >> > >> +#. Configure and install DPDK using Meson > >> + > >> + Meson is the preferred tool to build recent DPDK releases > >> + as Make support is deprecated and will be removed from DPDK 20.11. > >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > >> + > >> + Build and install the DPDK library:: > >> + > >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > >> + $ meson $DPDK_TARGET > >> + $ ninja -C $DPDK_TARGET > >> + $ sudo ninja -C $DPDK_TARGET install > >> + $ sudo ldconfig > >> + > >> + Detailed information can be found at `DPDK documentation`_. > >> + > >> #. (Optional) Configure DPDK as a shared library > >> > >> - DPDK can be built as either a static library or a shared library. By > >> - default, it is configured for the former. If you wish to use the latter, set > >> + When using Meson, DPDK is built both as static and shared library. > >> + So no extra configuration is required in this case. > >> + > >> + In case of Make, DPDK can be built as either a static library or a shared > >> + library. By default, it is configured for the former. If you wish to use > >> + the latter, set > >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in ``$DPDK_DIR/config/common_base``. > >> > >> .. note:: > >> @@ -87,7 +110,7 @@ Install DPDK > >> Minor performance loss is expected when using OVS with a shared DPDK > >> library compared to a static DPDK library. > >> > >> -#. Configure and install DPDK > >> +#. Configure and install DPDK using Make > >> > >> Build and install the DPDK library:: > >> > >> @@ -97,12 +120,22 @@ Install DPDK > >> > >> #. (Optional) Export the DPDK shared library location > >> > >> - If DPDK was built as a shared library, export the path to this library for > >> - use when building OVS:: > >> + If DPDK was built as a shared library using Make, export the path to this > >> + library for use when building OVS:: > >> > >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib > >> > >> + In case of Meson, exporting the path to library is not necessary if > >> + the DPDK libraries are system installed. For libraries installed using > >> + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this > >> + library and also update the $PKG_CONFIG_PATH for use before building OVS:: > >> + > >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > >> + > >> .. _DPDK sources: http://dpdk.org/rel > >> +.. _DPDK documentation: https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > >> > >> Install OVS > >> ~~~~~~~~~~~ > >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library (``--with-dpdk``). > >> > >> #. Bootstrap, if required, as described in :ref:`general-bootstrapping` > >> > >> -#. Configure the package using the ``--with-dpdk`` flag:: > >> +#. Configure the package using the ``--with-dpdk`` flag: > >> + > >> + Depending on the tool used to build DPDK and the type of > >> + DPDK library to use, one can configure OVS as follows: > >> + > >> + When DPDK is built using Meson, and OVS must consume DPDK shared libraries > >> + (also equivalent to leaving --with-dpdk option empty):: > >> + > >> + $ ./configure --with-dpdk=shared > > Hi all > > when I build ovs with --with-dpdk=shared, Compilation is fine, but we > > I launch the ovs: > > > > 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] > > -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > --socket-limit 1024,1024. > > 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' > > 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: > > No such device > > > > In dpdk: > > eal_plugins_init > > rte_pci_scan > > rte_bus_register > > eal_option_device_parse -- no pci bus > > > > pkg-config is pkgconf-1.4.2 > >> + When DPDK is built using Meson, and OVS must consume DPDK static libraries:: > >> + > >> + $ ./configure --with-dpdk=static > > Only one issue, if we use the pkgconfig-0.27 > > The ovs log: > > 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > 1024,1024. > > 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' > > 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > such device > > > > Because /bin/pkg-config --static --libs libdpdk > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > -l:librte_common_cpt.a -l:librte_common_dpaax.a > > -l:librte_common_iavf.a -l:librte_common_octeontx.a > > -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > -l:librte_common_mlx5.a -l:librte_common_qat.a > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > > -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > > -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > > -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > > -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > > -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > > -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > > -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > > -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > > -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > > > > Note that: > > -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't work.) > > I guess we dont hope that, It should be: > > -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > > > update pkg-config to pkgconf-1.4.2 > > /bin/pkg-config --static --libs libdpdk > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > -l:librte_common_dpaax.a -l:librte_common_iavf.a > > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > > -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > > > ovs works fine. > > Hmm. Thanks for the report. > > It'll be great if you can test this with new version of this patch: > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1-git-send-email-ian.stokes@intel.com/ Hi Ilya I tested the patch https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1-git-send-email-ian.stokes@intel.com/ As I said if we update the pkg-conf to new version. and we config the ovs --with-dpdk=static, ovs works fine. but if we build ovs --with-dpdk=shared, ovs doesn't work, there is error log: 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6] -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit 1024,1024. 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse device "0000:82:00.0" 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse device '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-6]' 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize DPDK: No such device because pci bus was not registered. In dpdk: eal_plugins_init rte_pci_scan eal_option_device_parse -- no pci bus > Bruce, David, it looks like not all versions of pkg-config could be used. > Do you know what is the minimal required version or how to protect us from > this kind of issues? > > Best regards, Ilya Maximets.
> -----Original Message----- > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > Sent: Tuesday, December 15, 2020 12:20 PM > To: Ilya Maximets <i.maximets@ovn.org> > Cc: Pai G, Sunil <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; > Richardson, Bruce <bruce.richardson@intel.com>; David Marchand > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > On Tue, Dec 15, 2020 at 5:32 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > > > On 12/15/20 5:02 AM, Tonghao Zhang wrote: > > > On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> > wrote: > > >> > > >> Make based build is deprecated in DPDK. Meson based > > >> build to be used for future DPDK releases. > > >> > > >> This updates travis, configure script and documentation > > >> for using DPDK Meson with OVS. > > >> > > >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs- > copy/builds/723510063 > > >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > > >> --- > > >> v3->v4: > > >> - Fix checkpatch errors > > >> > > >> v2->v3: > > >> - Update Documentation for vhost-user > > >> > > >> v1->v2: > > >> - Update Documentation > > >> - Simplify the pkg-config parsing script > > >> - Rename and move the pkg-config parsing script to python dir > > >> - Update travis to: > > >> - install DPDK to cached dir > > >> - disable DPDK tests > > >> - removed fPIC flag for DPDK > > >> - removed cross compilation for aarch64 > > >> --- > > >> .travis.yml | 3 ++ > > >> .travis/linux-build.sh | 39 ++++++++++----- > > >> .travis/linux-prepare.sh | 1 + > > >> Documentation/intro/install/afxdp.rst | 2 +- > > >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++-- > -- > > >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ > > >> acinclude.m4 | 44 +++++++++++------ > > >> python/automake.mk | 3 +- > > >> python/build/pkgcfg.py | 30 +++++++++++ > > >> 9 files changed, 149 insertions(+), 54 deletions(-) > > >> create mode 100644 python/build/pkgcfg.py > > >> > > >> diff --git a/.travis.yml b/.travis.yml > > >> index 3dd5d1d23..a8f9a4d79 100644 > > >> --- a/.travis.yml > > >> +++ b/.travis.yml > > >> @@ -27,6 +27,9 @@ addons: > > >> - selinux-policy-dev > > >> - libunbound-dev > > >> - libunwind-dev > > >> + - python3-setuptools > > >> + - python3-wheel > > >> + - ninja-build > > >> > > >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > > >> > > >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > > >> index 817bf24aa..14ef833c9 100755 > > >> --- a/.travis/linux-build.sh > > >> +++ b/.travis/linux-build.sh > > >> @@ -85,17 +85,29 @@ function install_dpdk() > > >> { > > >> local DPDK_VER=$1 > > >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > > >> + local DPDK_OPTS="" > > >> + local DPDK_LIB="" > > >> > > >> if [ -z "$TRAVIS_ARCH" ] || > > >> [ "$TRAVIS_ARCH" == "amd64" ]; then > > >> - TARGET="x86_64-native-linuxapp-gcc" > > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > > >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > > >> - TARGET="arm64-armv8a-linuxapp-gcc" > > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > > >> else > > >> echo "Target is unknown" > > >> exit 1 > > >> fi > > >> > > >> + if [ "$DPDK_SHARED" ]; then > > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > > >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > > >> + else > > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > > >> + fi > > >> + > > >> + # Export the following path for pkg-config to find the .pc file. > > >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > > >> + > > >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > > >> # Avoid using cache for git tree build. > > >> rm -rf dpdk-dir > > >> @@ -108,7 +120,8 @@ function install_dpdk() > > >> if [ -f "${VERSION_FILE}" ]; then > > >> VER=$(cat ${VERSION_FILE}) > > >> if [ "${VER}" = "${DPDK_VER}" ]; then > > >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk- > dir/build" > > >> + # Update the library paths. > > >> + sudo ldconfig > > >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk- > dir" > > >> return > > >> fi > > >> @@ -122,16 +135,20 @@ function install_dpdk() > > >> pushd dpdk-dir > > >> fi > > >> > > >> - make config CC=gcc T=$TARGET > > >> + # Disable building DPDK unit tests. Not needed for OVS build or > tests. > > >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > > >> > > >> - if [ "$DPDK_SHARED" ]; then > > >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' > build/.config > > >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > > >> - fi > > >> + # Install DPDK using prefix. > > >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > > >> + > > >> + CC=gcc meson $DPDK_OPTS build > > >> + ninja -C build > > >> + sudo ninja -C build install > > >> + > > >> + # Update the library paths. > > >> + sudo ldconfig > > >> > > >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > > >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > > >> - echo "Installed DPDK source in $(pwd)" > > >> + echo "Installed DPDK source" > > >> popd > > >> echo "${DPDK_VER}" > ${VERSION_FILE} > > >> } > > >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > > >> index 71eb347e8..1baa11641 100755 > > >> --- a/.travis/linux-prepare.sh > > >> +++ b/.travis/linux-prepare.sh > > >> @@ -22,6 +22,7 @@ cd .. > > >> > > >> pip3 install --disable-pip-version-check --user flake8 hacking > > >> pip3 install --user --upgrade docutils > > >> +pip3 install --user 'meson==0.47.1' > > >> > > >> if [ "$M32" ]; then > > >> # Installing 32-bit libraries. > > >> diff --git a/Documentation/intro/install/afxdp.rst > b/Documentation/intro/install/afxdp.rst > > >> index 3c8f78825..327f2b3df 100644 > > >> --- a/Documentation/intro/install/afxdp.rst > > >> +++ b/Documentation/intro/install/afxdp.rst > > >> @@ -396,7 +396,7 @@ PVP using vhostuser device > > >> -------------------------- > > >> First, build OVS with DPDK and AFXDP:: > > >> > > >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> > > >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > > >> make -j4 && make install > > >> > > >> Create a vhost-user port from OVS:: > > >> diff --git a/Documentation/intro/install/dpdk.rst > b/Documentation/intro/install/dpdk.rst > > >> index 39544f835..cd7e51c75 100644 > > >> --- a/Documentation/intro/install/dpdk.rst > > >> +++ b/Documentation/intro/install/dpdk.rst > > >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > requirements`_. > > >> .. _DPDK supported NIC: http://dpdk.org/doc/nics > > >> .. _DPDK requirements: > http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > >> > > >> +.. _dpdk-install: > > >> + > > >> Installing > > >> ---------- > > >> > > >> @@ -76,10 +78,31 @@ Install DPDK > > >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > > >> $ cd $DPDK_DIR > > >> > > >> +#. Configure and install DPDK using Meson > > >> + > > >> + Meson is the preferred tool to build recent DPDK releases > > >> + as Make support is deprecated and will be removed from DPDK > 20.11. > > >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > > >> + > > >> + Build and install the DPDK library:: > > >> + > > >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > > >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > > >> + $ meson $DPDK_TARGET > > >> + $ ninja -C $DPDK_TARGET > > >> + $ sudo ninja -C $DPDK_TARGET install > > >> + $ sudo ldconfig > > >> + > > >> + Detailed information can be found at `DPDK documentation`_. > > >> + > > >> #. (Optional) Configure DPDK as a shared library > > >> > > >> - DPDK can be built as either a static library or a shared library. > By > > >> - default, it is configured for the former. If you wish to use the > latter, set > > >> + When using Meson, DPDK is built both as static and shared > library. > > >> + So no extra configuration is required in this case. > > >> + > > >> + In case of Make, DPDK can be built as either a static library or > a shared > > >> + library. By default, it is configured for the former. If you > wish to use > > >> + the latter, set > > >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > ``$DPDK_DIR/config/common_base``. > > >> > > >> .. note:: > > >> @@ -87,7 +110,7 @@ Install DPDK > > >> Minor performance loss is expected when using OVS with a > shared DPDK > > >> library compared to a static DPDK library. > > >> > > >> -#. Configure and install DPDK > > >> +#. Configure and install DPDK using Make > > >> > > >> Build and install the DPDK library:: > > >> > > >> @@ -97,12 +120,22 @@ Install DPDK > > >> > > >> #. (Optional) Export the DPDK shared library location > > >> > > >> - If DPDK was built as a shared library, export the path to this > library for > > >> - use when building OVS:: > > >> + If DPDK was built as a shared library using Make, export the path > to this > > >> + library for use when building OVS:: > > >> > > >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp- > gcc/lib > > >> > > >> + In case of Meson, exporting the path to library is not necessary > if > > >> + the DPDK libraries are system installed. For libraries installed > using > > >> + a prefix(assuming $DPDK_INSTALL in the below case), export the > path to this > > >> + library and also update the $PKG_CONFIG_PATH for use before > building OVS:: > > >> + > > >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > > >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > > >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > > >> + > > >> .. _DPDK sources: http://dpdk.org/rel > > >> +.. _DPDK documentation: > https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > > >> > > >> Install OVS > > >> ~~~~~~~~~~~ > > >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK > library (``--with-dpdk``). > > >> > > >> #. Bootstrap, if required, as described in :ref:`general- > bootstrapping` > > >> > > >> -#. Configure the package using the ``--with-dpdk`` flag:: > > >> +#. Configure the package using the ``--with-dpdk`` flag: > > >> + > > >> + Depending on the tool used to build DPDK and the type of > > >> + DPDK library to use, one can configure OVS as follows: > > >> + > > >> + When DPDK is built using Meson, and OVS must consume DPDK shared > libraries > > >> + (also equivalent to leaving --with-dpdk option empty):: > > >> + > > >> + $ ./configure --with-dpdk=shared > > > Hi all > > > when I build ovs with --with-dpdk=shared, Compilation is fine, but we > > > I launch the ovs: > > > > > > 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > tor=[0-6] > > > -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > > --socket-limit 1024,1024. > > > 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > > > "0000:82:00.0" > > > 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > ntor=[0-6]' > > > 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: > > > No such device > > > > > > In dpdk: > > > eal_plugins_init > > > rte_pci_scan > > > rte_bus_register > > > eal_option_device_parse -- no pci bus > > > > > > pkg-config is pkgconf-1.4.2 > > >> + When DPDK is built using Meson, and OVS must consume DPDK static > libraries:: > > >> + > > >> + $ ./configure --with-dpdk=static > > > Only one issue, if we use the pkgconfig-0.27 > > > The ovs log: > > > 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > tor=[0-6] > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > > 1024,1024. > > > 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > > > "0000:82:00.0" > > > 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > ntor=[0-6]' > > > 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > > such device > > > > > > Because /bin/pkg-config --static --libs libdpdk > > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > > -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > > -l:librte_common_cpt.a -l:librte_common_dpaax.a > > > -l:librte_common_iavf.a -l:librte_common_octeontx.a > > > -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > > -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > > -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > > -l:librte_common_mlx5.a -l:librte_common_qat.a > > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > > -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > > > -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > > > -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > > > -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > > > -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > > > -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > > > -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > > > -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > > > -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > > > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > > > -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > > > > > > Note that: > > > -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't > work.) > > > I guess we dont hope that, It should be: > > > -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > > > > > update pkg-config to pkgconf-1.4.2 > > > /bin/pkg-config --static --libs libdpdk > > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > > -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > > -l:librte_common_dpaax.a -l:librte_common_iavf.a > > > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > > > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > > > -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > > > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > > > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > > > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > > > > > ovs works fine. > > > > Hmm. Thanks for the report. > > > > It'll be great if you can test this with new version of this patch: > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877- > 1-git-send-email-ian.stokes@intel.com/ > Hi Ilya > I tested the patch > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1- > git-send-email-ian.stokes@intel.com/ > As I said if we update the pkg-conf to new version. and we config the > ovs --with-dpdk=static, ovs works fine. > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is error > log: > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > tor=[0-6] > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > 1024,1024. > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse device > "0000:82:00.0" > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse device > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > ntor=[0-6]' > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize DPDK: No > such device > > because pci bus was not registered. > > In dpdk: > eal_plugins_init > rte_pci_scan > eal_option_device_parse -- no pci bus > > Is DPDK installed systemwide so that the drivers - including pci bus driver - can be picked up automatically on init? If not, you may well need to pass the path to the drivers via the "-d" flag on startup. Regards, /Bruce
On Tue, Dec 15, 2020 at 8:25 PM Richardson, Bruce <bruce.richardson@intel.com> wrote: > > > > > -----Original Message----- > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > Sent: Tuesday, December 15, 2020 12:20 PM > > To: Ilya Maximets <i.maximets@ovn.org> > > Cc: Pai G, Sunil <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; > > Richardson, Bruce <bruce.richardson@intel.com>; David Marchand > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > > meson build. > > > > On Tue, Dec 15, 2020 at 5:32 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > > > > > On 12/15/20 5:02 AM, Tonghao Zhang wrote: > > > > On Thu, Sep 3, 2020 at 2:06 AM Sunil Pai G <sunil.pai.g@intel.com> > > wrote: > > > >> > > > >> Make based build is deprecated in DPDK. Meson based > > > >> build to be used for future DPDK releases. > > > >> > > > >> This updates travis, configure script and documentation > > > >> for using DPDK Meson with OVS. > > > >> > > > >> Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs- > > copy/builds/723510063 > > > >> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> > > > >> --- > > > >> v3->v4: > > > >> - Fix checkpatch errors > > > >> > > > >> v2->v3: > > > >> - Update Documentation for vhost-user > > > >> > > > >> v1->v2: > > > >> - Update Documentation > > > >> - Simplify the pkg-config parsing script > > > >> - Rename and move the pkg-config parsing script to python dir > > > >> - Update travis to: > > > >> - install DPDK to cached dir > > > >> - disable DPDK tests > > > >> - removed fPIC flag for DPDK > > > >> - removed cross compilation for aarch64 > > > >> --- > > > >> .travis.yml | 3 ++ > > > >> .travis/linux-build.sh | 39 ++++++++++----- > > > >> .travis/linux-prepare.sh | 1 + > > > >> Documentation/intro/install/afxdp.rst | 2 +- > > > >> Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++-- > > -- > > > >> Documentation/topics/dpdk/vhost-user.rst | 18 +------ > > > >> acinclude.m4 | 44 +++++++++++------ > > > >> python/automake.mk | 3 +- > > > >> python/build/pkgcfg.py | 30 +++++++++++ > > > >> 9 files changed, 149 insertions(+), 54 deletions(-) > > > >> create mode 100644 python/build/pkgcfg.py > > > >> > > > >> diff --git a/.travis.yml b/.travis.yml > > > >> index 3dd5d1d23..a8f9a4d79 100644 > > > >> --- a/.travis.yml > > > >> +++ b/.travis.yml > > > >> @@ -27,6 +27,9 @@ addons: > > > >> - selinux-policy-dev > > > >> - libunbound-dev > > > >> - libunwind-dev > > > >> + - python3-setuptools > > > >> + - python3-wheel > > > >> + - ninja-build > > > >> > > > >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > > > >> > > > >> diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > > > >> index 817bf24aa..14ef833c9 100755 > > > >> --- a/.travis/linux-build.sh > > > >> +++ b/.travis/linux-build.sh > > > >> @@ -85,17 +85,29 @@ function install_dpdk() > > > >> { > > > >> local DPDK_VER=$1 > > > >> local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" > > > >> + local DPDK_OPTS="" > > > >> + local DPDK_LIB="" > > > >> > > > >> if [ -z "$TRAVIS_ARCH" ] || > > > >> [ "$TRAVIS_ARCH" == "amd64" ]; then > > > >> - TARGET="x86_64-native-linuxapp-gcc" > > > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu > > > >> elif [ "$TRAVIS_ARCH" == "aarch64" ]; then > > > >> - TARGET="arm64-armv8a-linuxapp-gcc" > > > >> + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu > > > >> else > > > >> echo "Target is unknown" > > > >> exit 1 > > > >> fi > > > >> > > > >> + if [ "$DPDK_SHARED" ]; then > > > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" > > > >> + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > > > >> + else > > > >> + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" > > > >> + fi > > > >> + > > > >> + # Export the following path for pkg-config to find the .pc file. > > > >> + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > > > >> + > > > >> if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then > > > >> # Avoid using cache for git tree build. > > > >> rm -rf dpdk-dir > > > >> @@ -108,7 +120,8 @@ function install_dpdk() > > > >> if [ -f "${VERSION_FILE}" ]; then > > > >> VER=$(cat ${VERSION_FILE}) > > > >> if [ "${VER}" = "${DPDK_VER}" ]; then > > > >> - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk- > > dir/build" > > > >> + # Update the library paths. > > > >> + sudo ldconfig > > > >> echo "Found cached DPDK ${VER} build in $(pwd)/dpdk- > > dir" > > > >> return > > > >> fi > > > >> @@ -122,16 +135,20 @@ function install_dpdk() > > > >> pushd dpdk-dir > > > >> fi > > > >> > > > >> - make config CC=gcc T=$TARGET > > > >> + # Disable building DPDK unit tests. Not needed for OVS build or > > tests. > > > >> + DPDK_OPTS="$DPDK_OPTS -Dtests=false" > > > >> > > > >> - if [ "$DPDK_SHARED" ]; then > > > >> - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' > > build/.config > > > >> - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib > > > >> - fi > > > >> + # Install DPDK using prefix. > > > >> + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" > > > >> + > > > >> + CC=gcc meson $DPDK_OPTS build > > > >> + ninja -C build > > > >> + sudo ninja -C build install > > > >> + > > > >> + # Update the library paths. > > > >> + sudo ldconfig > > > >> > > > >> - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' > > > >> - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" > > > >> - echo "Installed DPDK source in $(pwd)" > > > >> + echo "Installed DPDK source" > > > >> popd > > > >> echo "${DPDK_VER}" > ${VERSION_FILE} > > > >> } > > > >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > > > >> index 71eb347e8..1baa11641 100755 > > > >> --- a/.travis/linux-prepare.sh > > > >> +++ b/.travis/linux-prepare.sh > > > >> @@ -22,6 +22,7 @@ cd .. > > > >> > > > >> pip3 install --disable-pip-version-check --user flake8 hacking > > > >> pip3 install --user --upgrade docutils > > > >> +pip3 install --user 'meson==0.47.1' > > > >> > > > >> if [ "$M32" ]; then > > > >> # Installing 32-bit libraries. > > > >> diff --git a/Documentation/intro/install/afxdp.rst > > b/Documentation/intro/install/afxdp.rst > > > >> index 3c8f78825..327f2b3df 100644 > > > >> --- a/Documentation/intro/install/afxdp.rst > > > >> +++ b/Documentation/intro/install/afxdp.rst > > > >> @@ -396,7 +396,7 @@ PVP using vhostuser device > > > >> -------------------------- > > > >> First, build OVS with DPDK and AFXDP:: > > > >> > > > >> - ./configure --enable-afxdp --with-dpdk=<dpdk path> > > > >> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> > > > >> make -j4 && make install > > > >> > > > >> Create a vhost-user port from OVS:: > > > >> diff --git a/Documentation/intro/install/dpdk.rst > > b/Documentation/intro/install/dpdk.rst > > > >> index 39544f835..cd7e51c75 100644 > > > >> --- a/Documentation/intro/install/dpdk.rst > > > >> +++ b/Documentation/intro/install/dpdk.rst > > > >> @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK > > requirements`_. > > > >> .. _DPDK supported NIC: http://dpdk.org/doc/nics > > > >> .. _DPDK requirements: > > http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html > > > >> > > > >> +.. _dpdk-install: > > > >> + > > > >> Installing > > > >> ---------- > > > >> > > > >> @@ -76,10 +78,31 @@ Install DPDK > > > >> $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 > > > >> $ cd $DPDK_DIR > > > >> > > > >> +#. Configure and install DPDK using Meson > > > >> + > > > >> + Meson is the preferred tool to build recent DPDK releases > > > >> + as Make support is deprecated and will be removed from DPDK > > 20.11. > > > >> + OVS supports DPDK Meson builds from DPDK 19.11 onwards. > > > >> + > > > >> + Build and install the DPDK library:: > > > >> + > > > >> + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > >> + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > > > >> + $ meson $DPDK_TARGET > > > >> + $ ninja -C $DPDK_TARGET > > > >> + $ sudo ninja -C $DPDK_TARGET install > > > >> + $ sudo ldconfig > > > >> + > > > >> + Detailed information can be found at `DPDK documentation`_. > > > >> + > > > >> #. (Optional) Configure DPDK as a shared library > > > >> > > > >> - DPDK can be built as either a static library or a shared library. > > By > > > >> - default, it is configured for the former. If you wish to use the > > latter, set > > > >> + When using Meson, DPDK is built both as static and shared > > library. > > > >> + So no extra configuration is required in this case. > > > >> + > > > >> + In case of Make, DPDK can be built as either a static library or > > a shared > > > >> + library. By default, it is configured for the former. If you > > wish to use > > > >> + the latter, set > > > >> ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in > > ``$DPDK_DIR/config/common_base``. > > > >> > > > >> .. note:: > > > >> @@ -87,7 +110,7 @@ Install DPDK > > > >> Minor performance loss is expected when using OVS with a > > shared DPDK > > > >> library compared to a static DPDK library. > > > >> > > > >> -#. Configure and install DPDK > > > >> +#. Configure and install DPDK using Make > > > >> > > > >> Build and install the DPDK library:: > > > >> > > > >> @@ -97,12 +120,22 @@ Install DPDK > > > >> > > > >> #. (Optional) Export the DPDK shared library location > > > >> > > > >> - If DPDK was built as a shared library, export the path to this > > library for > > > >> - use when building OVS:: > > > >> + If DPDK was built as a shared library using Make, export the path > > to this > > > >> + library for use when building OVS:: > > > >> > > > >> $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp- > > gcc/lib > > > >> > > > >> + In case of Meson, exporting the path to library is not necessary > > if > > > >> + the DPDK libraries are system installed. For libraries installed > > using > > > >> + a prefix(assuming $DPDK_INSTALL in the below case), export the > > path to this > > > >> + library and also update the $PKG_CONFIG_PATH for use before > > building OVS:: > > > >> + > > > >> + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu > > > >> + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH > > > >> + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH > > > >> + > > > >> .. _DPDK sources: http://dpdk.org/rel > > > >> +.. _DPDK documentation: > > https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html > > > >> > > > >> Install OVS > > > >> ~~~~~~~~~~~ > > > >> @@ -121,17 +154,27 @@ has to be configured to build against the DPDK > > library (``--with-dpdk``). > > > >> > > > >> #. Bootstrap, if required, as described in :ref:`general- > > bootstrapping` > > > >> > > > >> -#. Configure the package using the ``--with-dpdk`` flag:: > > > >> +#. Configure the package using the ``--with-dpdk`` flag: > > > >> + > > > >> + Depending on the tool used to build DPDK and the type of > > > >> + DPDK library to use, one can configure OVS as follows: > > > >> + > > > >> + When DPDK is built using Meson, and OVS must consume DPDK shared > > libraries > > > >> + (also equivalent to leaving --with-dpdk option empty):: > > > >> + > > > >> + $ ./configure --with-dpdk=shared > > > > Hi all > > > > when I build ovs with --with-dpdk=shared, Compilation is fine, but we > > > > I launch the ovs: > > > > > > > > 41 2020-12-14T10:42:22.964Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > tor=[0-6] > > > > -c 0xe --huge-dir /dev/hugepages --socket -mem 1024,1024 > > > > --socket-limit 1024,1024. > > > > 42 2020-12-14T10:42:22.969Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > 43 2020-12-14T10:42:22.969Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > > 44 2020-12-14T10:42:22.969Z|00021|dpdk|ERR|EAL: failed to parse device > > > > "0000:82:00.0" > > > > 45 2020-12-14T10:42:22.969Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > ntor=[0-6]' > > > > 46 2020-12-14T10:42:22.969Z|00023|dpdk|EMER|Unable to initialize DPDK: > > > > No such device > > > > > > > > In dpdk: > > > > eal_plugins_init > > > > rte_pci_scan > > > > rte_bus_register > > > > eal_option_device_parse -- no pci bus > > > > > > > > pkg-config is pkgconf-1.4.2 > > > >> + When DPDK is built using Meson, and OVS must consume DPDK static > > libraries:: > > > >> + > > > >> + $ ./configure --with-dpdk=static > > > > Only one issue, if we use the pkgconfig-0.27 > > > > The ovs log: > > > > 2020-12-14T11:58:22.622Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > tor=[0-6] > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > > > 1024,1024. > > > > 2020-12-14T11:58:22.627Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > 2020-12-14T11:58:22.627Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > > 2020-12-14T11:58:22.627Z|00021|dpdk|ERR|EAL: failed to parse device > > > > "0000:82:00.0" > > > > 2020-12-14T11:58:22.627Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > ntor=[0-6]' > > > > 2020-12-14T11:58:22.627Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > > > such device > > > > > > > > Because /bin/pkg-config --static --libs libdpdk > > > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive -Wl,--no-whole-archive > > > > -Wl,--as-needed -pthread -L/root/local/dpdk-next-net/lib64 > > > > -l:librte_common_cpt.a -l:librte_common_dpaax.a > > > > -l:librte_common_iavf.a -l:librte_common_octeontx.a > > > > -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a > > > > -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a > > > > -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a > > > > -l:librte_common_mlx5.a -l:librte_common_qat.a > > > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > > > -l:librte_kvargs.a -lmlx5 -libverbs -lrte_node -lrte_graph -lrte_bpf > > > > -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib > > > > -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched > > > > -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump > > > > -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni > > > > -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev > > > > -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev > > > > -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer > > > > -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > > > > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > > > > -lrte_eal -lrte_telemetry -lrte_kvargs -lm -ldl -lnuma -lpcap ' > > > > > > > > Note that: > > > > -Wl,--whole-archive -Wl,--no-whole-archive (RTE_INIT in dpdk doesn't > > work.) > > > > I guess we dont hope that, It should be: > > > > -Wl,--whole-archive ...(other -lxxx)... -Wl,--no-whole-archive > > > > > > > > update pkg-config to pkgconf-1.4.2 > > > > /bin/pkg-config --static --libs libdpdk > > > > pkg_cv_DPDK_LIBS='-Wl,--whole-archive > > > > -L/root/local/dpdk-next-net/lib64 -l:librte_common_cpt.a > > > > -l:librte_common_dpaax.a -l:librte_common_iavf.a > > > > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > > > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > > > > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > > > > -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a > > > > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > > > > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > > > > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > > > > -l:librte_mempool_stack.a -l:librte_net_af_packet.a > > > > -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a > > > > -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a > > > > -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a > > > > -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a > > > > -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a > > > > -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a > > > > -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a > > > > -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a > > > > -l:librte_net_memif.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a > > > > -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a > > > > -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a > > > > -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a > > > > -l:librte_net_softnic.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a > > > > -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a > > > > -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a > > > > -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > > > -l:librte_compress_octeontx.a -l:librte_regex_mlx5.a > > > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a > > > > -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a > > > > -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a > > > > -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a > > > > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > > > > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > > > > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > > > > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > > > > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > > > > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > > > > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > > > > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > > > > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > > > > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > > > > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > > > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lmlx5 -libverbs > > > > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > > > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > > > > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > > > > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > > > > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > > > > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > > > > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > > > > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > > > > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > > > > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > > > > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap ' > > > > > > > > ovs works fine. > > > > > > Hmm. Thanks for the report. > > > > > > It'll be great if you can test this with new version of this patch: > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877- > > 1-git-send-email-ian.stokes@intel.com/ > > Hi Ilya > > I tested the patch > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825-30877-1- > > git-send-email-ian.stokes@intel.com/ > > As I said if we update the pkg-conf to new version. and we config the > > ovs --with-dpdk=static, ovs works fine. > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is error > > log: > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > tor=[0-6] > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > 1024,1024. > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse device > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > ntor=[0-6]' > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > such device > > > > because pci bus was not registered. > > > > In dpdk: > > eal_plugins_init > > rte_pci_scan > > eal_option_device_parse -- no pci bus > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > driver - can be picked up automatically on init? If not, you may well > need to pass the path to the drivers via the "-d" flag on startup. Hi I build ovs, shown as below: export DPDK_TARGET=x86_64-native-linuxapp-gcc meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 -Dbuildtype=debug -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* $DPDK_TARGET ninja -C $DPDK_TARGET install ldconfig export LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64:$LD_LIBRARY_PATH export PKG_CONFIG_PATH=/root/local/dpdk-next-net/lib64/pkgconfig/:$PKG_CONFIG_PATH ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 -Wno-deprecated-declarations' make && make install One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > Regards, > /Bruce
<snip> > > > Hi Ilya > > > I tested the patch > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > 30877-1- > > > git-send-email-ian.stokes@intel.com/ > > > As I said if we update the pkg-conf to new version. and we config the > > > ovs --with-dpdk=static, ovs works fine. > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > error > > > log: > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > tor=[0-6] > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > > 1024,1024. > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse device > > > "0000:82:00.0" > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > ntor=[0-6]' > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > > such device > > > > > > because pci bus was not registered. > > > > > > In dpdk: > > > eal_plugins_init > > > rte_pci_scan > > > eal_option_device_parse -- no pci bus > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > driver - can be picked up automatically on init? If not, you may well > > need to pass the path to the drivers via the "-d" flag on startup. > Hi > I build ovs, shown as below: > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > -Dbuildtype=debug > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > $DPDK_TARGET > ninja -C $DPDK_TARGET install > ldconfig > > export LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64:$LD_LIBRARY_PATH > export PKG_CONFIG_PATH=/root/local/dpdk-next- > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > -Wno-deprecated-declarations' > make && make install > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > Regards, > > /Bruce In the case above, I think the drivers should all be loaded automatically from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The RTE_EAL_PMD_PATH should point to that location. To verify, can you check the "rte_build_config.h" file in your DPDK build, and confirm that the path given there for RTE_EAL_PMD_PATH corresponds to the path where the actual driver .so files are installed? /Bruce
Hi Zhang, Could you give this patch a try as well ? http://patchwork.ozlabs.org/project/openvswitch/patch/20201215164128.13153-1-sunil.pai.g@intel.com/ This has a check in place to throw an error if the pkg-config is buggy. Note that this is for the master branch. So, when you switch from dpdk-latest to master , you would need to execute ./boot.sh before running the configure command. > -----Original Message----- > From: Richardson, Bruce <bruce.richardson@intel.com> > Sent: Tuesday, December 15, 2020 7:09 PM > To: Tonghao Zhang <xiangxia.m.yue@gmail.com> > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > <snip> > > > > > Hi Ilya > > > > I tested the patch > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > 30877-1- > > > > git-send-email-ian.stokes@intel.com/ > > > > As I said if we update the pkg-conf to new version. and we config > > > > the ovs --with-dpdk=static, ovs works fine. > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > > error > > > > log: > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd - > a > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > epr > > esen > > > > tor=[0-6] > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > > > --socket-limit 1024,1024. > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > > > nodes > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > > device "0000:82:00.0" > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > > device > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > ep > > rese > > > > ntor=[0-6]' > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > > DPDK: No such device > > > > > > > > because pci bus was not registered. > > > > > > > > In dpdk: > > > > eal_plugins_init > > > > rte_pci_scan > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > > driver - can be picked up automatically on init? If not, you may > > > well need to pass the path to the drivers via the "-d" flag on startup. > > Hi > > I build ovs, shown as below: > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > -Dbuildtype=debug > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > $DPDK_TARGET > > ninja -C $DPDK_TARGET install > > ldconfig > > > > export > > LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64:$LD_LIBRARY_PATH > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > -Wno-deprecated-declarations' > > make && make install > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > Regards, > > > /Bruce > > In the case above, I think the drivers should all be loaded automatically from > "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > RTE_EAL_PMD_PATH should point to that location. > > To verify, can you check the "rte_build_config.h" file in your DPDK build, and > confirm that the path given there for RTE_EAL_PMD_PATH corresponds to > the path where the actual driver .so files are installed? > > /Bruce Thanks and regards Sunil
On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce <bruce.richardson@intel.com> wrote: > > <snip> > > > > > Hi Ilya > > > > I tested the patch > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > 30877-1- > > > > git-send-email-ian.stokes@intel.com/ > > > > As I said if we update the pkg-conf to new version. and we config the > > > > ovs --with-dpdk=static, ovs works fine. > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > > error > > > > log: > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > > tor=[0-6] > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket-limit > > > > 1024,1024. > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA nodes > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse device > > > > "0000:82:00.0" > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse device > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > > ntor=[0-6]' > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize DPDK: No > > > > such device > > > > > > > > because pci bus was not registered. > > > > > > > > In dpdk: > > > > eal_plugins_init > > > > rte_pci_scan > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > > driver - can be picked up automatically on init? If not, you may well > > > need to pass the path to the drivers via the "-d" flag on startup. > > Hi > > I build ovs, shown as below: > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > -Dbuildtype=debug > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > $DPDK_TARGET > > ninja -C $DPDK_TARGET install > > ldconfig > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64:$LD_LIBRARY_PATH > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > -Wno-deprecated-declarations' > > make && make install > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > Regards, > > > /Bruce > > In the case above, I think the drivers should all be loaded automatically > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The RTE_EAL_PMD_PATH > should point to that location. > > To verify, can you check the "rte_build_config.h" file in your DPDK build, > and confirm that the path given there for RTE_EAL_PMD_PATH corresponds to > the path where the actual driver .so files are installed? In the rte_build_config.h: #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1" I use dpdk-next-net repo to build dpdk, the last commit is d3fa7b89f0b893f051cb4cef1cbc961cb59f5721 > /Bruce
On Wed, Dec 16, 2020 at 12:55 AM Pai G, Sunil <sunil.pai.g@intel.com> wrote: > > Hi Zhang, > > Could you give this patch a try as well ? > http://patchwork.ozlabs.org/project/openvswitch/patch/20201215164128.13153-1-sunil.pai.g@intel.com/ > This has a check in place to throw an error if the pkg-config is buggy. > > Note that this is for the master branch. So, when you switch from dpdk-latest to master , you would need to execute ./boot.sh before running the configure command. Hi Pai Building the ovs --with-dpdk=static, works fine for us. so Tested-by: Tonghao Zhang <xiangxia.m.yue@gmail.com> By the way, ovs --with-dpdk=shared, still doesn't work. > > -----Original Message----- > > From: Richardson, Bruce <bruce.richardson@intel.com> > > Sent: Tuesday, December 15, 2020 7:09 PM > > To: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > > meson build. > > > > <snip> > > > > > > > Hi Ilya > > > > > I tested the patch > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > 30877-1- > > > > > git-send-email-ian.stokes@intel.com/ > > > > > As I said if we update the pkg-conf to new version. and we config > > > > > the ovs --with-dpdk=static, ovs works fine. > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > > > error > > > > > log: > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd - > > a > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > epr > > > esen > > > > > tor=[0-6] > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > > > > --socket-limit 1024,1024. > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > > > > nodes > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > > > device "0000:82:00.0" > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > > > device > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > ep > > > rese > > > > > ntor=[0-6]' > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > > > DPDK: No such device > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > In dpdk: > > > > > eal_plugins_init > > > > > rte_pci_scan > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > > > driver - can be picked up automatically on init? If not, you may > > > > well need to pass the path to the drivers via the "-d" flag on startup. > > > Hi > > > I build ovs, shown as below: > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > -Dbuildtype=debug > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > $DPDK_TARGET > > > ninja -C $DPDK_TARGET install > > > ldconfig > > > > > > export > > > LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64:$LD_LIBRARY_PATH > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > -Wno-deprecated-declarations' > > > make && make install > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > > > Regards, > > > > /Bruce > > > > In the case above, I think the drivers should all be loaded automatically from > > "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > > RTE_EAL_PMD_PATH should point to that location. > > > > To verify, can you check the "rte_build_config.h" file in your DPDK build, and > > confirm that the path given there for RTE_EAL_PMD_PATH corresponds to > > the path where the actual driver .so files are installed? > > > > /Bruce > > > Thanks and regards > Sunil
Hi Zhang, Please see comments inline. > -----Original Message----- > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > Sent: Wednesday, December 16, 2020 7:56 AM > To: Pai G, Sunil <sunil.pai.g@intel.com> > Cc: Ilya Maximets <i.maximets@ovn.org>; ovs dev <dev@openvswitch.org>; > David Marchand <david.marchand@redhat.com>; Stokes, Ian > <ian.stokes@intel.com>; Richardson, Bruce <bruce.richardson@intel.com> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > On Wed, Dec 16, 2020 at 12:55 AM Pai G, Sunil <sunil.pai.g@intel.com> > wrote: > > > > Hi Zhang, > > > > Could you give this patch a try as well ? > > > http://patchwork.ozlabs.org/project/openvswitch/patch/20201215164128.1 > > 3153-1-sunil.pai.g@intel.com/ This has a check in place to throw an > > error if the pkg-config is buggy. > > > > Note that this is for the master branch. So, when you switch from dpdk- > latest to master , you would need to execute ./boot.sh before running the > configure command. > Hi Pai > Building the ovs --with-dpdk=static, works fine for us. so > Tested-by: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > By the way, ovs --with-dpdk=shared, still doesn't work. I tried your build commands using latest dpdk-next-net and OVS-dpdk-latest branch. It worked fine for me on ubuntu18.04. Moreover, GHA and travis CI's are passing for the shared builds. So, it might be a problem with the system you are using. Best to check the paths you are setting as well. Btw, which branch in OVS and which OS are you using ? Could you also try running a sample application in dpdk like l2fwd and check if it works fine as well ? > > > > -----Original Message----- > > > From: Richardson, Bruce <bruce.richardson@intel.com> > > > Sent: Tuesday, December 15, 2020 7:09 PM > > > To: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David > > > Marchand <david.marchand@redhat.com>; Stokes, Ian > > > <ian.stokes@intel.com> > > > Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for > > > DPDK meson build. > > > > > > <snip> > > > > > > > > > Hi Ilya > > > > > > I tested the patch > > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977 > > > > > > 825- > > > > 30877-1- > > > > > > git-send-email-ian.stokes@intel.com/ > > > > > > As I said if we update the pkg-conf to new version. and we > > > > > > config the ovs --with-dpdk=static, ovs works fine. > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, > > > > > > there is > > > > error > > > > > > log: > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: > > > > > > ovs-vswitchd - > > > a > > > > > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > > epr > > > > esen > > > > > > tor=[0-6] > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > > > > > --socket-limit 1024,1024. > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 > > > > > > lcore(s) > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 > NUMA > > > > > > nodes > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > > > > device "0000:82:00.0" > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > > > > device > > > > > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > > ep > > > > rese > > > > > > ntor=[0-6]' > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > > > > DPDK: No such device > > > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > > > In dpdk: > > > > > > eal_plugins_init > > > > > > rte_pci_scan > > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci > > > > > bus driver - can be picked up automatically on init? If not, you > > > > > may well need to pass the path to the drivers via the "-d" flag on > startup. > > > > Hi > > > > I build ovs, shown as below: > > > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > > -Dbuildtype=debug > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > > $DPDK_TARGET > > > > ninja -C $DPDK_TARGET install > > > > ldconfig > > > > > > > > export > > > > LD_LIBRARY_PATH=/root/local/dpdk-next- > net/lib64:$LD_LIBRARY_PATH > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > > -Wno-deprecated-declarations' > > > > make && make install > > > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > > > > > Regards, > > > > > /Bruce > > > > > > In the case above, I think the drivers should all be loaded > > > automatically from > > > "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > RTE_EAL_PMD_PATH should point to that location. > > > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > > > build, and confirm that the path given there for RTE_EAL_PMD_PATH > > > corresponds to the path where the actual driver .so files are installed? > > > > > > /Bruce > > > > > > Thanks and regards > > Sunil > > > > -- > Best regards, Tonghao
On Wed, Dec 16, 2020 at 8:43 PM Pai G, Sunil <sunil.pai.g@intel.com> wrote: > > Hi Zhang, > Please see comments inline. > > > > -----Original Message----- > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > Sent: Wednesday, December 16, 2020 7:56 AM > > To: Pai G, Sunil <sunil.pai.g@intel.com> > > Cc: Ilya Maximets <i.maximets@ovn.org>; ovs dev <dev@openvswitch.org>; > > David Marchand <david.marchand@redhat.com>; Stokes, Ian > > <ian.stokes@intel.com>; Richardson, Bruce <bruce.richardson@intel.com> > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > > meson build. > > > > On Wed, Dec 16, 2020 at 12:55 AM Pai G, Sunil <sunil.pai.g@intel.com> > > wrote: > > > > > > Hi Zhang, > > > > > > Could you give this patch a try as well ? > > > > > http://patchwork.ozlabs.org/project/openvswitch/patch/20201215164128.1 > > > 3153-1-sunil.pai.g@intel.com/ This has a check in place to throw an > > > error if the pkg-config is buggy. > > > > > > Note that this is for the master branch. So, when you switch from dpdk- > > latest to master , you would need to execute ./boot.sh before running the > > configure command. > > Hi Pai > > Building the ovs --with-dpdk=static, works fine for us. so > > Tested-by: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > > > By the way, ovs --with-dpdk=shared, still doesn't work. > > I tried your build commands using latest dpdk-next-net and OVS-dpdk-latest branch. It worked fine for me on ubuntu18.04. > Moreover, GHA and travis CI's are passing for the shared builds. > So, it might be a problem with the system you are using. > Best to check the paths you are setting as well. > > Btw, which branch in OVS and which OS are you using ? > Could you also try running a sample application in dpdk like l2fwd and check if it works fine as well ? Yes, I use the dpdk helloworld to test. ./build/helloworld -n 4 -c 4 -a 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,representor=[0-2] EAL: Detected 56 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: Probing VFIO support... EAL: VFIO support initialized EAL: Probe PCI driver: mlx5_pci (15b3:101d) device: 0000:82:00.0 (socket 1) common_mlx5: RTE_MEM is selected. mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000. EAL: No legacy callbacks, legacy socket not created hello from core 2 But there is error in ovs: $ ovs-vsctl show a360a9ac-d269-4e89-b01c-c09331b9fb00 Bridge br-int datapath_type: netdev Port tun Interface tun type: vxlan options: {key=flow, local_ip="10.96.74.2", remote_ip=flow} Port br-int Interface br-int type: internal Port rep1 Interface rep1 type: dpdk options: {dpdk-devargs="0000:82:00.0,representor=[1]"} error: "Error attaching device '0000:82:00.0,representor=[1]' to DPDK" ovs-vswitchd log: 2020-12-15T21:20:41.703Z|00333|netdev|WARN|rep1: could not set configuration (Invalid argument) 2020-12-15T21:20:41.703Z|00334|dpdk|ERR|Invalid port_id=256 2020-12-15T21:20:41.703Z|00335|dpdk|ERR|EAL: failed to parse device "0000:82:00.0" 2020-12-15T21:20:41.703Z|00336|dpdk|ERR|EAL: failed to parse device "0000:82:00.0" 2020-12-15T21:20:41.703Z|00337|dpdk|ERR|EAL: Failed to attach device on primary process 2020-12-15T21:20:41.703Z|00338|netdev_dpdk|WARN|Error attaching device '0000:82:00.0,representor=[0]' to DPDK One question, did you attach a dpdk port to ovs ? ovs commit-id: af06184705072804a4c1374f9c824c9e4c241c26 dpdk-next-net d3fa7b89f0b893f051cb4cef1cbc961cb59f5721 systeminfo CentOS Linux release 7.6.1810 (Core) kernel 3.10.0-957.1.3.el7.x86_64 > > > > > > > -----Original Message----- > > > > From: Richardson, Bruce <bruce.richardson@intel.com> > > > > Sent: Tuesday, December 15, 2020 7:09 PM > > > > To: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David > > > > Marchand <david.marchand@redhat.com>; Stokes, Ian > > > > <ian.stokes@intel.com> > > > > Subject: RE: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for > > > > DPDK meson build. > > > > > > > > <snip> > > > > > > > > > > > Hi Ilya > > > > > > > I tested the patch > > > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977 > > > > > > > 825- > > > > > 30877-1- > > > > > > > git-send-email-ian.stokes@intel.com/ > > > > > > > As I said if we update the pkg-conf to new version. and we > > > > > > > config the ovs --with-dpdk=static, ovs works fine. > > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, > > > > > > > there is > > > > > error > > > > > > > log: > > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: > > > > > > > ovs-vswitchd - > > > > a > > > > > > > > > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > > > epr > > > > > esen > > > > > > > tor=[0-6] > > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 > > > > > > > --socket-limit 1024,1024. > > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 > > > > > > > lcore(s) > > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 > > NUMA > > > > > > > nodes > > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > > > > > device "0000:82:00.0" > > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > > > > > device > > > > > > > > > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > > > ep > > > > > rese > > > > > > > ntor=[0-6]' > > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > > > > > DPDK: No such device > > > > > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > > > > > In dpdk: > > > > > > > eal_plugins_init > > > > > > > rte_pci_scan > > > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci > > > > > > bus driver - can be picked up automatically on init? If not, you > > > > > > may well need to pass the path to the drivers via the "-d" flag on > > startup. > > > > > Hi > > > > > I build ovs, shown as below: > > > > > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > > > -Dbuildtype=debug > > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > > > $DPDK_TARGET > > > > > ninja -C $DPDK_TARGET install > > > > > ldconfig > > > > > > > > > > export > > > > > LD_LIBRARY_PATH=/root/local/dpdk-next- > > net/lib64:$LD_LIBRARY_PATH > > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > > > -Wno-deprecated-declarations' > > > > > make && make install > > > > > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > > > > > > > Regards, > > > > > > /Bruce > > > > > > > > In the case above, I think the drivers should all be loaded > > > > automatically from > > > > "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > > RTE_EAL_PMD_PATH should point to that location. > > > > > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > > > > build, and confirm that the path given there for RTE_EAL_PMD_PATH > > > > corresponds to the path where the actual driver .so files are installed? > > > > > > > > /Bruce > > > > > > > > > Thanks and regards > > > Sunil > > > > > > > > -- > > Best regards, Tonghao
<snipped> > > I tried your build commands using latest dpdk-next-net and OVS-dpdk- > latest branch. It worked fine for me on ubuntu18.04. > > Moreover, GHA and travis CI's are passing for the shared builds. > > So, it might be a problem with the system you are using. > > Best to check the paths you are setting as well. > > > > Btw, which branch in OVS and which OS are you using ? > > Could you also try running a sample application in dpdk like l2fwd and check > if it works fine as well ? > Yes, I use the dpdk helloworld to test. > ./build/helloworld -n 4 -c 4 -a > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > epresentor=[0-2] > EAL: Detected 56 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: Probing VFIO support... > EAL: VFIO support initialized > EAL: Probe PCI driver: mlx5_pci (15b3:101d) device: 0000:82:00.0 (socket 1) > common_mlx5: RTE_MEM is selected. > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000. > EAL: No legacy callbacks, legacy socket not created hello from core 2 > > But there is error in ovs: > $ ovs-vsctl show > a360a9ac-d269-4e89-b01c-c09331b9fb00 > Bridge br-int > datapath_type: netdev > Port tun > Interface tun > type: vxlan > options: {key=flow, local_ip="10.96.74.2", remote_ip=flow} > Port br-int > Interface br-int > type: internal > Port rep1 > Interface rep1 > type: dpdk > options: {dpdk-devargs="0000:82:00.0,representor=[1]"} > error: "Error attaching device '0000:82:00.0,representor=[1]' to DPDK" > > > ovs-vswitchd log: > 2020-12-15T21:20:41.703Z|00333|netdev|WARN|rep1: could not set > configuration (Invalid argument) 2020-12- > 15T21:20:41.703Z|00334|dpdk|ERR|Invalid port_id=256 > 2020-12-15T21:20:41.703Z|00335|dpdk|ERR|EAL: failed to parse device > "0000:82:00.0" > 2020-12-15T21:20:41.703Z|00336|dpdk|ERR|EAL: failed to parse device > "0000:82:00.0" > 2020-12-15T21:20:41.703Z|00337|dpdk|ERR|EAL: Failed to attach device on > primary process 2020-12- > 15T21:20:41.703Z|00338|netdev_dpdk|WARN|Error attaching device > '0000:82:00.0,representor=[0]' to DPDK > > One question, did you attach a dpdk port to ovs ? Yes, I attached physical as well as virtual ports and both got added without err. I suspect you might have to add a conf file pointing to the dpdk libraries , something like mentioned here: https://www.cyberciti.biz/faq/linux-setting-changing-library-path/ Also, what does the ldd report for ovs-vswitchd binary ? > > ovs commit-id: > af06184705072804a4c1374f9c824c9e4c241c26 > dpdk-next-net > d3fa7b89f0b893f051cb4cef1cbc961cb59f5721 > systeminfo > CentOS Linux release 7.6.1810 (Core) > kernel 3.10.0-957.1.3.el7.x86_64 > > > > > > <snipped>
> -----Original Message----- > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > Sent: Wednesday, December 16, 2020 1:58 AM > To: Richardson, Bruce <bruce.richardson@intel.com> > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce > <bruce.richardson@intel.com> wrote: > > > > <snip> > > > > > > > Hi Ilya > > > > > I tested the patch > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > 30877-1- > > > > > git-send-email-ian.stokes@intel.com/ > > > > > As I said if we update the pkg-conf to new version. and we config > the > > > > > ovs --with-dpdk=static, ovs works fine. > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > > > error > > > > > log: > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > > > tor=[0-6] > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket- > limit > > > > > 1024,1024. > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > nodes > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > device > > > > > "0000:82:00.0" > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > device > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > > > ntor=[0-6]' > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > DPDK: No > > > > > such device > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > In dpdk: > > > > > eal_plugins_init > > > > > rte_pci_scan > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > > > driver - can be picked up automatically on init? If not, you may > well > > > > need to pass the path to the drivers via the "-d" flag on startup. > > > Hi > > > I build ovs, shown as below: > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > -Dbuildtype=debug > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > $DPDK_TARGET > > > ninja -C $DPDK_TARGET install > > > ldconfig > > > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next- > net/lib64:$LD_LIBRARY_PATH > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > -Wno-deprecated-declarations' > > > make && make install > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > > > Regards, > > > > /Bruce > > > > In the case above, I think the drivers should all be loaded > automatically > > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > RTE_EAL_PMD_PATH > > should point to that location. > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > build, > > and confirm that the path given there for RTE_EAL_PMD_PATH corresponds > to > > the path where the actual driver .so files are installed? > In the rte_build_config.h: > #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1" > That looks a correct path. Can you confirm that the actual pmd .so files are in that directory on your system?
On Wed, Dec 16, 2020 at 9:34 PM Pai G, Sunil <sunil.pai.g@intel.com> wrote: > > <snipped> > > > I tried your build commands using latest dpdk-next-net and OVS-dpdk- > > latest branch. It worked fine for me on ubuntu18.04. > > > Moreover, GHA and travis CI's are passing for the shared builds. > > > So, it might be a problem with the system you are using. > > > Best to check the paths you are setting as well. > > > > > > Btw, which branch in OVS and which OS are you using ? > > > Could you also try running a sample application in dpdk like l2fwd and check > > if it works fine as well ? > > Yes, I use the dpdk helloworld to test. > > ./build/helloworld -n 4 -c 4 -a > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,r > > epresentor=[0-2] > > EAL: Detected 56 lcore(s) > > EAL: Detected 2 NUMA nodes > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > EAL: Selected IOVA mode 'VA' > > EAL: Probing VFIO support... > > EAL: VFIO support initialized > > EAL: Probe PCI driver: mlx5_pci (15b3:101d) device: 0000:82:00.0 (socket 1) > > common_mlx5: RTE_MEM is selected. > > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000. > > EAL: No legacy callbacks, legacy socket not created hello from core 2 > > > > But there is error in ovs: > > $ ovs-vsctl show > > a360a9ac-d269-4e89-b01c-c09331b9fb00 > > Bridge br-int > > datapath_type: netdev > > Port tun > > Interface tun > > type: vxlan > > options: {key=flow, local_ip="10.96.74.2", remote_ip=flow} > > Port br-int > > Interface br-int > > type: internal > > Port rep1 > > Interface rep1 > > type: dpdk > > options: {dpdk-devargs="0000:82:00.0,representor=[1]"} > > error: "Error attaching device '0000:82:00.0,representor=[1]' to DPDK" > > > > > > ovs-vswitchd log: > > 2020-12-15T21:20:41.703Z|00333|netdev|WARN|rep1: could not set > > configuration (Invalid argument) 2020-12- > > 15T21:20:41.703Z|00334|dpdk|ERR|Invalid port_id=256 > > 2020-12-15T21:20:41.703Z|00335|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 2020-12-15T21:20:41.703Z|00336|dpdk|ERR|EAL: failed to parse device > > "0000:82:00.0" > > 2020-12-15T21:20:41.703Z|00337|dpdk|ERR|EAL: Failed to attach device on > > primary process 2020-12- > > 15T21:20:41.703Z|00338|netdev_dpdk|WARN|Error attaching device > > '0000:82:00.0,representor=[0]' to DPDK > > > > One question, did you attach a dpdk port to ovs ? > > Yes, I attached physical as well as virtual ports and both got added without err. > > I suspect you might have to add a conf file pointing to the dpdk libraries , something like mentioned here: > https://www.cyberciti.biz/faq/linux-setting-changing-library-path/ still not work I export LD_LIBRARY_PATH=/root/local/dpdk-next-net/lib64 change that to: $ cat /etc/ld.so.conf.d/dpdk.conf /root/local/dpdk-next-net/lib64 $ ldconfig > > Also, what does the ldd report for ovs-vswitchd binary ? linux-vdso.so.1 => (0x00007fff52364000) /usr/local/gundam/gundam_client/preload/$LIB/gundam_preload.so => /usr/local/gundam/gundam_client/preload/lib64/gundam_preload.so (0x00007fa24dda2000) librte_vhost.so.21 => /root/local/dpdk-next-net/lib64/librte_vhost.so.21 (0x00007fa24dab9000) librte_ethdev.so.21 => /root/local/dpdk-next-net/lib64/librte_ethdev.so.21 (0x00007fa24d479000) librte_meter.so.21 => /root/local/dpdk-next-net/lib64/librte_meter.so.21 (0x00007fa24d276000) librte_mbuf.so.21 => /root/local/dpdk-next-net/lib64/librte_mbuf.so.21 (0x00007fa24d05b000) librte_mempool.so.21 => /root/local/dpdk-next-net/lib64/librte_mempool.so.21 (0x00007fa24ce4f000) librte_eal.so.21 => /root/local/dpdk-next-net/lib64/librte_eal.so.21 (0x00007fa24cb50000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa24c934000) librt.so.1 => /lib64/librt.so.1 (0x00007fa24c72c000) libm.so.6 => /lib64/libm.so.6 (0x00007fa24c42a000) libunbound.so.2 => /lib64/libunbound.so.2 (0x00007fa24c129000) libc.so.6 => /lib64/libc.so.6 (0x00007fa24bd5b000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fa24bb57000) libnuma.so.1 => /lib64/libnuma.so.1 (0x00007fa24b94b000) librte_kvargs.so.21 => /root/local/dpdk-next-net/lib64/librte_kvargs.so.21 (0x00007fa24b748000) librte_telemetry.so.21 => /root/local/dpdk-next-net/lib64/librte_telemetry.so.21 (0x00007fa24b53e000) librte_net.so.21 => /root/local/dpdk-next-net/lib64/librte_net.so.21 (0x00007fa24b32f000) librte_ring.so.21 => /root/local/dpdk-next-net/lib64/librte_ring.so.21 (0x00007fa24b12b000) librte_cryptodev.so.21 => /root/local/dpdk-next-net/lib64/librte_cryptodev.so.21 (0x00007fa24af13000) librte_hash.so.21 => /root/local/dpdk-next-net/lib64/librte_hash.so.21 (0x00007fa24acf7000) librte_rcu.so.21 => /root/local/dpdk-next-net/lib64/librte_rcu.so.21 (0x00007fa24aaed000) librte_pci.so.21 => /root/local/dpdk-next-net/lib64/librte_pci.so.21 (0x00007fa24a8ea000) /lib64/ld-linux-x86-64.so.2 (0x00007fa24dfab000) libssl.so.10 => /lib64/libssl.so.10 (0x00007fa24a678000) libevent-2.0.so.5 => /lib64/libevent-2.0.so.5 (0x00007fa24a430000) libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007fa24a064000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fa249c01000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa2499eb000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fa24979e000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fa2494b5000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fa2492b1000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fa249096000) libz.so.1 => /lib64/libz.so.1 (0x00007fa248e80000) libutil.so.1 => /lib64/libutil.so.1 (0x00007fa248c7d000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fa248a6e000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fa24886a000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fa248650000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fa248429000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fa2481c7000) > > > > > > ovs commit-id: > > af06184705072804a4c1374f9c824c9e4c241c26 > > dpdk-next-net > > d3fa7b89f0b893f051cb4cef1cbc961cb59f5721 > > systeminfo > > CentOS Linux release 7.6.1810 (Core) > > kernel 3.10.0-957.1.3.el7.x86_64 > > > > > > > > > > <snipped>
On Wed, Dec 16, 2020 at 9:41 PM Richardson, Bruce <bruce.richardson@intel.com> wrote: > > > > > -----Original Message----- > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > Sent: Wednesday, December 16, 2020 1:58 AM > > To: Richardson, Bruce <bruce.richardson@intel.com> > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > > meson build. > > > > On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce > > <bruce.richardson@intel.com> wrote: > > > > > > <snip> > > > > > > > > > Hi Ilya > > > > > > I tested the patch > > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > > 30877-1- > > > > > > git-send-email-ian.stokes@intel.com/ > > > > > > As I said if we update the pkg-conf to new version. and we config > > the > > > > > > ovs --with-dpdk=static, ovs works fine. > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, there is > > > > error > > > > > > log: > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs-vswitchd -a > > > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > > > > tor=[0-6] > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 --socket- > > limit > > > > > > 1024,1024. > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 lcore(s) > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > nodes > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > device > > > > > > "0000:82:00.0" > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > device > > > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > > > > ntor=[0-6]' > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > DPDK: No > > > > > > such device > > > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > > > In dpdk: > > > > > > eal_plugins_init > > > > > > rte_pci_scan > > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci bus > > > > > driver - can be picked up automatically on init? If not, you may > > well > > > > > need to pass the path to the drivers via the "-d" flag on startup. > > > > Hi > > > > I build ovs, shown as below: > > > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > > -Dbuildtype=debug > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > > $DPDK_TARGET > > > > ninja -C $DPDK_TARGET install > > > > ldconfig > > > > > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next- > > net/lib64:$LD_LIBRARY_PATH > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > > -Wno-deprecated-declarations' > > > > make && make install > > > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag ? > > > > > > > > > Regards, > > > > > /Bruce > > > > > > In the case above, I think the drivers should all be loaded > > automatically > > > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > > RTE_EAL_PMD_PATH > > > should point to that location. > > > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > > build, > > > and confirm that the path given there for RTE_EAL_PMD_PATH corresponds > > to > > > the path where the actual driver .so files are installed? > > In the rte_build_config.h: > > #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1" > > > That looks a correct path. Can you confirm that the actual pmd .so files are > in that directory on your system? Yes, librte_net_mlx5.so librte_common_mlx5.so But note that there is not mlx5 so the ldd output ldd /root/local/openvswitch-2.14/sbin/ovs-vswitchd | grep -i mlx5 I debugged it and found that pci was not in bus_list, so pci device was not attached to ovs.
> -----Original Message----- > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > Sent: Wednesday, December 16, 2020 1:49 PM > To: Richardson, Bruce <bruce.richardson@intel.com> > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > meson build. > > On Wed, Dec 16, 2020 at 9:41 PM Richardson, Bruce > <bruce.richardson@intel.com> wrote: > > > > > > > > > -----Original Message----- > > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > > Sent: Wednesday, December 16, 2020 1:58 AM > > > To: Richardson, Bruce <bruce.richardson@intel.com> > > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for > DPDK > > > meson build. > > > > > > On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce > > > <bruce.richardson@intel.com> wrote: > > > > > > > > <snip> > > > > > > > > > > > Hi Ilya > > > > > > > I tested the patch > > > > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > > > 30877-1- > > > > > > > git-send-email-ian.stokes@intel.com/ > > > > > > > As I said if we update the pkg-conf to new version. and we > config > > > the > > > > > > > ovs --with-dpdk=static, ovs works fine. > > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, > there is > > > > > error > > > > > > > log: > > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs- > vswitchd -a > > > > > > > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > > > > > tor=[0-6] > > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 -- > socket- > > > limit > > > > > > > 1024,1024. > > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 > lcore(s) > > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > > nodes > > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > device > > > > > > > "0000:82:00.0" > > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > device > > > > > > > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > > > > > ntor=[0-6]' > > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > DPDK: No > > > > > > > such device > > > > > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > > > > > In dpdk: > > > > > > > eal_plugins_init > > > > > > > rte_pci_scan > > > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci > bus > > > > > > driver - can be picked up automatically on init? If not, you may > > > well > > > > > > need to pass the path to the drivers via the "-d" flag on > startup. > > > > > Hi > > > > > I build ovs, shown as below: > > > > > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > > > -Dbuildtype=debug > > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > > > $DPDK_TARGET > > > > > ninja -C $DPDK_TARGET install > > > > > ldconfig > > > > > > > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next- > > > net/lib64:$LD_LIBRARY_PATH > > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > > > -Wno-deprecated-declarations' > > > > > make && make install > > > > > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag > ? > > > > > > > > > > > Regards, > > > > > > /Bruce > > > > > > > > In the case above, I think the drivers should all be loaded > > > automatically > > > > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > > > RTE_EAL_PMD_PATH > > > > should point to that location. > > > > > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > > > build, > > > > and confirm that the path given there for RTE_EAL_PMD_PATH > corresponds > > > to > > > > the path where the actual driver .so files are installed? > > > In the rte_build_config.h: > > > #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds- > 21.1" > > > > > That looks a correct path. Can you confirm that the actual pmd .so files > are > > in that directory on your system? > Yes, > librte_net_mlx5.so > librte_common_mlx5.so > > But note that there is not mlx5 so the ldd output > ldd /root/local/openvswitch-2.14/sbin/ovs-vswitchd | grep -i mlx5 > > I debugged it and found that pci was not in bus_list, so pci device > was not attached to ovs. Does librte_bus_pci.so.21 exist in "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1" directory too? Also can you turn on debug logging for EAL, which should show the individual drivers getting loaded at runtime. The loading is done by function "eal_plugins_init" in DPDK. /Bruce
On Wed, Dec 16, 2020 at 9:57 PM Richardson, Bruce <bruce.richardson@intel.com> wrote: > > > > > -----Original Message----- > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > Sent: Wednesday, December 16, 2020 1:49 PM > > To: Richardson, Bruce <bruce.richardson@intel.com> > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for DPDK > > meson build. > > > > On Wed, Dec 16, 2020 at 9:41 PM Richardson, Bruce > > <bruce.richardson@intel.com> wrote: > > > > > > > > > > > > > -----Original Message----- > > > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com> > > > > Sent: Wednesday, December 16, 2020 1:58 AM > > > > To: Richardson, Bruce <bruce.richardson@intel.com> > > > > Cc: Ilya Maximets <i.maximets@ovn.org>; Pai G, Sunil > > > > <sunil.pai.g@intel.com>; ovs dev <dev@openvswitch.org>; David Marchand > > > > <david.marchand@redhat.com>; Stokes, Ian <ian.stokes@intel.com> > > > > Subject: Re: [ovs-dev] [PATCH dpdk-latest v4] build: Add support for > > DPDK > > > > meson build. > > > > > > > > On Tue, Dec 15, 2020 at 9:38 PM Richardson, Bruce > > > > <bruce.richardson@intel.com> wrote: > > > > > > > > > > <snip> > > > > > > > > > > > > > Hi Ilya > > > > > > > > I tested the patch > > > > > > > > > > https://patchwork.ozlabs.org/project/openvswitch/patch/1607977825- > > > > > > 30877-1- > > > > > > > > git-send-email-ian.stokes@intel.com/ > > > > > > > > As I said if we update the pkg-conf to new version. and we > > config > > > > the > > > > > > > > ovs --with-dpdk=static, ovs works fine. > > > > > > > > but if we build ovs --with-dpdk=shared, ovs doesn't work, > > there is > > > > > > error > > > > > > > > log: > > > > > > > > 2020-12-14T20:15:07.940Z|00018|dpdk|INFO|EAL ARGS: ovs- > > vswitchd -a > > > > > > > > > > > > > > > > > > > > 0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represen > > > > > > > > tor=[0-6] > > > > > > > > -c 0xe --huge-dir /dev/hugepages --socket-mem 1024,1024 -- > > socket- > > > > limit > > > > > > > > 1024,1024. > > > > > > > > 2020-12-14T20:15:07.946Z|00019|dpdk|INFO|EAL: Detected 56 > > lcore(s) > > > > > > > > 2020-12-14T20:15:07.946Z|00020|dpdk|INFO|EAL: Detected 2 NUMA > > > > nodes > > > > > > > > 2020-12-14T20:15:07.946Z|00021|dpdk|ERR|EAL: failed to parse > > > > device > > > > > > > > "0000:82:00.0" > > > > > > > > 2020-12-14T20:15:07.946Z|00022|dpdk|ERR|EAL: Unable to parse > > > > device > > > > > > > > > > > > > > > > > > > > '0000:82:00.0,dv_flow_en=1,dv_esw_en=1,l3_vxlan_en=1,dv_xmeta_en=2,represe > > > > > > > > ntor=[0-6]' > > > > > > > > 2020-12-14T20:15:07.946Z|00023|dpdk|EMER|Unable to initialize > > > > DPDK: No > > > > > > > > such device > > > > > > > > > > > > > > > > because pci bus was not registered. > > > > > > > > > > > > > > > > In dpdk: > > > > > > > > eal_plugins_init > > > > > > > > rte_pci_scan > > > > > > > > eal_option_device_parse -- no pci bus > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Is DPDK installed systemwide so that the drivers - including pci > > bus > > > > > > > driver - can be picked up automatically on init? If not, you may > > > > well > > > > > > > need to pass the path to the drivers via the "-d" flag on > > startup. > > > > > > Hi > > > > > > I build ovs, shown as below: > > > > > > > > > > > > export DPDK_TARGET=x86_64-native-linuxapp-gcc > > > > > > meson -Dprefix=/root/local/dpdk-next-net -Dmax_ethports=256 > > > > > > -Dbuildtype=debug > > > > > > -Ddisable_drivers=event/*,net/tap,net/nfb,net/mlx4,crypto/* > > > > > > $DPDK_TARGET > > > > > > ninja -C $DPDK_TARGET install > > > > > > ldconfig > > > > > > > > > > > > export LD_LIBRARY_PATH=/root/local/dpdk-next- > > > > net/lib64:$LD_LIBRARY_PATH > > > > > > export PKG_CONFIG_PATH=/root/local/dpdk-next- > > > > > > net/lib64/pkgconfig/:$PKG_CONFIG_PATH > > > > > > ./configure --enable-Werror --prefix=/root/local/openvswitch-2.14 > > > > > > --disable-ssl --disable-libcapng --with-dpdk=shared 'CFLAGS=-g -O0 > > > > > > -Wno-deprecated-declarations' > > > > > > make && make install > > > > > > > > > > > > One question if we export LD_LIBRARY_PATH, we also use "-d" flag > > ? > > > > > > > > > > > > > Regards, > > > > > > > /Bruce > > > > > > > > > > In the case above, I think the drivers should all be loaded > > > > automatically > > > > > from "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.0/". The > > > > RTE_EAL_PMD_PATH > > > > > should point to that location. > > > > > > > > > > To verify, can you check the "rte_build_config.h" file in your DPDK > > > > build, > > > > > and confirm that the path given there for RTE_EAL_PMD_PATH > > corresponds > > > > to > > > > > the path where the actual driver .so files are installed? > > > > In the rte_build_config.h: > > > > #define RTE_EAL_PMD_PATH "/root/local/dpdk-next-net/lib64/dpdk/pmds- > > 21.1" > > > > > > > That looks a correct path. Can you confirm that the actual pmd .so files > > are > > > in that directory on your system? > > Yes, > > librte_net_mlx5.so > > librte_common_mlx5.so > > > > But note that there is not mlx5 so the ldd output > > ldd /root/local/openvswitch-2.14/sbin/ovs-vswitchd | grep -i mlx5 > > > > I debugged it and found that pci was not in bus_list, so pci device > > was not attached to ovs. > > Does librte_bus_pci.so.21 exist in "/root/local/dpdk-next-net/lib64/dpdk/pmds-21.1" > directory too? > Also can you turn on debug logging for EAL, which should show the individual > drivers getting loaded at runtime. The loading is done by function "eal_plugins_init" > in DPDK. Hi I found the root cause, I changed the eal_plugins_init(but not commit this change when I build ovs with dpdk 20.11 using my personal patch). I changed this for a long time, and now I test the Pai patch still using the my dpdk patch. Sorry for that. > /Bruce
diff --git a/.travis.yml b/.travis.yml index 3dd5d1d23..a8f9a4d79 100644 --- a/.travis.yml +++ b/.travis.yml @@ -27,6 +27,9 @@ addons: - selinux-policy-dev - libunbound-dev - libunwind-dev + - python3-setuptools + - python3-wheel + - ninja-build before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index 817bf24aa..14ef833c9 100755 --- a/.travis/linux-build.sh +++ b/.travis/linux-build.sh @@ -85,17 +85,29 @@ function install_dpdk() { local DPDK_VER=$1 local VERSION_FILE="dpdk-dir/travis-dpdk-cache-version" + local DPDK_OPTS="" + local DPDK_LIB="" if [ -z "$TRAVIS_ARCH" ] || [ "$TRAVIS_ARCH" == "amd64" ]; then - TARGET="x86_64-native-linuxapp-gcc" + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/x86_64-linux-gnu elif [ "$TRAVIS_ARCH" == "aarch64" ]; then - TARGET="arm64-armv8a-linuxapp-gcc" + DPDK_LIB=$(pwd)/dpdk-dir/build/lib/aarch64-linux-gnu else echo "Target is unknown" exit 1 fi + if [ "$DPDK_SHARED" ]; then + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=shared" + export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH + else + EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=static" + fi + + # Export the following path for pkg-config to find the .pc file. + export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH + if [ "${DPDK_VER##refs/*/}" != "${DPDK_VER}" ]; then # Avoid using cache for git tree build. rm -rf dpdk-dir @@ -108,7 +120,8 @@ function install_dpdk() if [ -f "${VERSION_FILE}" ]; then VER=$(cat ${VERSION_FILE}) if [ "${VER}" = "${DPDK_VER}" ]; then - EXTRA_OPTS="${EXTRA_OPTS} --with-dpdk=$(pwd)/dpdk-dir/build" + # Update the library paths. + sudo ldconfig echo "Found cached DPDK ${VER} build in $(pwd)/dpdk-dir" return fi @@ -122,16 +135,20 @@ function install_dpdk() pushd dpdk-dir fi - make config CC=gcc T=$TARGET + # Disable building DPDK unit tests. Not needed for OVS build or tests. + DPDK_OPTS="$DPDK_OPTS -Dtests=false" - if [ "$DPDK_SHARED" ]; then - sed -i '/CONFIG_RTE_BUILD_SHARED_LIB=n/s/=n/=y/' build/.config - export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/$TARGET/lib - fi + # Install DPDK using prefix. + DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build" + + CC=gcc meson $DPDK_OPTS build + ninja -C build + sudo ninja -C build install + + # Update the library paths. + sudo ldconfig - make -j4 CC=gcc EXTRA_CFLAGS='-fPIC' - EXTRA_OPTS="$EXTRA_OPTS --with-dpdk=$(pwd)/build" - echo "Installed DPDK source in $(pwd)" + echo "Installed DPDK source" popd echo "${DPDK_VER}" > ${VERSION_FILE} } diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh index 71eb347e8..1baa11641 100755 --- a/.travis/linux-prepare.sh +++ b/.travis/linux-prepare.sh @@ -22,6 +22,7 @@ cd .. pip3 install --disable-pip-version-check --user flake8 hacking pip3 install --user --upgrade docutils +pip3 install --user 'meson==0.47.1' if [ "$M32" ]; then # Installing 32-bit libraries. diff --git a/Documentation/intro/install/afxdp.rst b/Documentation/intro/install/afxdp.rst index 3c8f78825..327f2b3df 100644 --- a/Documentation/intro/install/afxdp.rst +++ b/Documentation/intro/install/afxdp.rst @@ -396,7 +396,7 @@ PVP using vhostuser device -------------------------- First, build OVS with DPDK and AFXDP:: - ./configure --enable-afxdp --with-dpdk=<dpdk path> + ./configure --enable-afxdp --with-dpdk=shared|static|<dpdk path> make -j4 && make install Create a vhost-user port from OVS:: diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index 39544f835..cd7e51c75 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -62,6 +62,8 @@ Detailed system requirements can be found at `DPDK requirements`_. .. _DPDK supported NIC: http://dpdk.org/doc/nics .. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html +.. _dpdk-install: + Installing ---------- @@ -76,10 +78,31 @@ Install DPDK $ export DPDK_DIR=/usr/src/dpdk-stable-19.11.2 $ cd $DPDK_DIR +#. Configure and install DPDK using Meson + + Meson is the preferred tool to build recent DPDK releases + as Make support is deprecated and will be removed from DPDK 20.11. + OVS supports DPDK Meson builds from DPDK 19.11 onwards. + + Build and install the DPDK library:: + + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ meson $DPDK_TARGET + $ ninja -C $DPDK_TARGET + $ sudo ninja -C $DPDK_TARGET install + $ sudo ldconfig + + Detailed information can be found at `DPDK documentation`_. + #. (Optional) Configure DPDK as a shared library - DPDK can be built as either a static library or a shared library. By - default, it is configured for the former. If you wish to use the latter, set + When using Meson, DPDK is built both as static and shared library. + So no extra configuration is required in this case. + + In case of Make, DPDK can be built as either a static library or a shared + library. By default, it is configured for the former. If you wish to use + the latter, set ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in ``$DPDK_DIR/config/common_base``. .. note:: @@ -87,7 +110,7 @@ Install DPDK Minor performance loss is expected when using OVS with a shared DPDK library compared to a static DPDK library. -#. Configure and install DPDK +#. Configure and install DPDK using Make Build and install the DPDK library:: @@ -97,12 +120,22 @@ Install DPDK #. (Optional) Export the DPDK shared library location - If DPDK was built as a shared library, export the path to this library for - use when building OVS:: + If DPDK was built as a shared library using Make, export the path to this + library for use when building OVS:: $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib + In case of Meson, exporting the path to library is not necessary if + the DPDK libraries are system installed. For libraries installed using + a prefix(assuming $DPDK_INSTALL in the below case), export the path to this + library and also update the $PKG_CONFIG_PATH for use before building OVS:: + + $ export $DPDK_LIB=$DPDK_INSTALL/lib/x86_64-linux-gnu + $ export LD_LIBRARY_PATH=$DPDK_LIB/:$LD_LIBRARY_PATH + $ export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH + .. _DPDK sources: http://dpdk.org/rel +.. _DPDK documentation: https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html Install OVS ~~~~~~~~~~~ @@ -121,17 +154,27 @@ has to be configured to build against the DPDK library (``--with-dpdk``). #. Bootstrap, if required, as described in :ref:`general-bootstrapping` -#. Configure the package using the ``--with-dpdk`` flag:: +#. Configure the package using the ``--with-dpdk`` flag: + + Depending on the tool used to build DPDK and the type of + DPDK library to use, one can configure OVS as follows: + + When DPDK is built using Meson, and OVS must consume DPDK shared libraries + (also equivalent to leaving --with-dpdk option empty):: + + $ ./configure --with-dpdk=shared + + When DPDK is built using Meson, and OVS must consume DPDK static libraries:: + + $ ./configure --with-dpdk=static + + When DPDK is built using Make(for shared or static):: $ ./configure --with-dpdk=$DPDK_BUILD where ``DPDK_BUILD`` is the path to the built DPDK library. This can be skipped if DPDK library is installed in its default location. - If no path is provided to ``--with-dpdk``, but a pkg-config configuration - for libdpdk is available the include paths will be generated via an - equivalent ``pkg-config --cflags libdpdk``. - .. note:: While ``--with-dpdk`` is required, you can pass any other configuration option described in :ref:`general-configuring`. diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 4af738d11..da3c4c9fa 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -389,23 +389,7 @@ application in the VM. To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or :ref:`dpdk-vhost-user-client`. Once started, connect to the VM, download the -DPDK sources to VM and build DPDK:: - - $ cd /root/dpdk/ - $ wget https://fast.dpdk.org/rel/dpdk-19.11.2.tar.xz - $ tar xf dpdk-19.11.2.tar.xz - $ export DPDK_DIR=/root/dpdk/dpdk-stable-19.11.2 - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET - $ cd $DPDK_DIR - $ make install T=$DPDK_TARGET DESTDIR=install - -Build the test-pmd application:: - - $ cd app/test-pmd - $ export RTE_SDK=$DPDK_DIR - $ export RTE_TARGET=$DPDK_TARGET - $ make +DPDK sources to VM and build DPDK as described in :ref:`dpdk-install`. Setup huge pages and DPDK devices using UIO:: diff --git a/acinclude.m4 b/acinclude.m4 index 84f344da0..412b2dd55 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -334,8 +334,10 @@ dnl dnl Configure DPDK source tree AC_DEFUN([OVS_CHECK_DPDK], [ AC_ARG_WITH([dpdk], - [AC_HELP_STRING([--with-dpdk=/path/to/dpdk], - [Specify the DPDK build directory])], + [AC_HELP_STRING([--with-dpdk=static|shared|/path/to/dpdk], + [Specify "static" or "shared" depending on the + DPDK libraries to use only if built using Meson + OR the DPDK build directory in case of Make])], [have_dpdk=true]) AC_MSG_CHECKING([whether dpdk is enabled]) @@ -345,13 +347,24 @@ AC_DEFUN([OVS_CHECK_DPDK], [ else AC_MSG_RESULT([yes]) case "$with_dpdk" in - yes) + "shared" | "static" | "yes") DPDK_AUTO_DISCOVER="true" - PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ - DPDK_INCLUDE="$DPDK_CFLAGS" - DPDK_LIB="$DPDK_LIBS"], [ - DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" - DPDK_LIB="-ldpdk"]) + case "$with_dpdk" in + "shared" | "yes") + PKG_CHECK_MODULES([DPDK], [libdpdk], [ + DPDK_INCLUDE="$DPDK_CFLAGS" + DPDK_LIB="$DPDK_LIBS"], [ + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" + DPDK_LIB="-ldpdk"]) + ;; + "static") + PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ + DPDK_INCLUDE="$DPDK_CFLAGS" + DPDK_LIB="$DPDK_LIBS"], [ + DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" + DPDK_LIB="-ldpdk"]) + ;; + esac ;; *) DPDK_AUTO_DISCOVER="false" @@ -424,8 +437,9 @@ AC_DEFUN([OVS_CHECK_DPDK], [ [AC_MSG_RESULT([no]) if test "$DPDK_AUTO_DISCOVER" = "true"; then AC_MSG_ERROR(m4_normalize([ - Could not find DPDK library in default search path, Use --with-dpdk - to specify the DPDK library installed in non-standard location])) + Could not find DPDK library in default search path, update + PKG_CONFIG_PATH for pkg-config to find the .pc file in + non-standard location])) else AC_MSG_ERROR([Could not find DPDK libraries in $DPDK_LIB_DIR]) fi @@ -451,10 +465,12 @@ AC_DEFUN([OVS_CHECK_DPDK], [ # OTOH newer versions of dpdk pkg-config (generated with Meson) # will already have flagged just the right set of libs with # --whole-archive - in those cases do not wrap it once more. - case "$DPDK_LIB" in - *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; - *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive - esac + if [[ "$pkg_failed" != "no" ]];then + DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive + else + DPDK_vswitchd_LDFLAGS=`python3 ${srcdir}/python/build/pkgcfg.py $DPDK_LIB` + fi + AC_SUBST([DPDK_vswitchd_LDFLAGS]) AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) fi diff --git a/python/automake.mk b/python/automake.mk index 2f08c7701..69d9800f9 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -47,7 +47,8 @@ ovs_pyfiles = \ EXTRA_DIST += \ python/build/__init__.py \ python/build/nroff.py \ - python/build/soutil.py + python/build/soutil.py \ + python/build/pkgcfg.py # PyPI support. EXTRA_DIST += \ diff --git a/python/build/pkgcfg.py b/python/build/pkgcfg.py new file mode 100644 index 000000000..7cee3cb03 --- /dev/null +++ b/python/build/pkgcfg.py @@ -0,0 +1,30 @@ +# Copyright (c) 2020 Intel, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License") +# You may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# Distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# Limitations under the License. + +# The purpose of this script is to parse the libraries +# From pkg-config in case of DPDK Meson builds. + +import sys +def parse_pkg_cfg_libs(arg): + linker_prefix = "-Wl," + # Libtool expects libraries to be comma separated + # And -Wl must appear only once. + final_string = ','.join(map(str.strip,arg[1:])).replace('-Wl,','') + final_string = arg[0]+" "+linker_prefix+final_string + # Ld only understands -lpthread. + final_string = final_string.replace('-pthread','-lpthread') + return final_string + +if __name__ == "__main__": + print(parse_pkg_cfg_libs(sys.argv[1:]))
Make based build is deprecated in DPDK. Meson based build to be used for future DPDK releases. This updates travis, configure script and documentation for using DPDK Meson with OVS. Tested-at: https://travis-ci.org/github/Sunil-Pai-G/ovs-copy/builds/723510063 Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> --- v3->v4: - Fix checkpatch errors v2->v3: - Update Documentation for vhost-user v1->v2: - Update Documentation - Simplify the pkg-config parsing script - Rename and move the pkg-config parsing script to python dir - Update travis to: - install DPDK to cached dir - disable DPDK tests - removed fPIC flag for DPDK - removed cross compilation for aarch64 --- .travis.yml | 3 ++ .travis/linux-build.sh | 39 ++++++++++----- .travis/linux-prepare.sh | 1 + Documentation/intro/install/afxdp.rst | 2 +- Documentation/intro/install/dpdk.rst | 63 ++++++++++++++++++++---- Documentation/topics/dpdk/vhost-user.rst | 18 +------ acinclude.m4 | 44 +++++++++++------ python/automake.mk | 3 +- python/build/pkgcfg.py | 30 +++++++++++ 9 files changed, 149 insertions(+), 54 deletions(-) create mode 100644 python/build/pkgcfg.py