Message ID | CAHJXk3YnBgWbAKQTo7A3UjnerVS-zvDRHKQjeo-__DS=Ld=vBQ@mail.gmail.com |
---|---|
State | Handled Elsewhere |
Headers | show |
Series | [ovs-dev] build: check libibverbs deps before linking with dpdk | expand |
Context | Check | Description |
---|---|---|
ovsrobot/apply-robot | success | apply and check: success |
ovsrobot/github-robot-_Build_and_Test | fail | github build: failed |
On 11/10/21 05:10, Harold Huang wrote: > When I build ovs-dpdk without mlx4/mlx5 driver, the configure log is as > follows: > > checking for DPDK... yes > checking for faulty pkg-config version... no > checking for rte_config.h... yes > checking whether RTE_LIBRTE_VHOST_NUMA is declared... yes > checking for library containing get_mempolicy... -lnuma > checking whether RTE_EAL_NUMA_AWARE_HUGEPAGES is declared... yes > checking for library containing get_mempolicy... (cached) -lnuma > checking whether RTE_LIBRTE_PMD_PCAP is declared... yes > checking for library containing pcap_dump_close... -lpcap > checking whether RTE_LIBRTE_PMD_AF_XDP is declared... no > checking whether RTE_LIBRTE_VHOST_NUMA is declared... (cached) yes > checking whether RTE_LIBRTE_MLX5_PMD is declared... no > checking whether RTE_LIBRTE_MLX4_PMD is declared... no > checking for library containing dlopen... -ldl > checking whether linking with dpdk works... no > configure: error: Could not find DPDK library in default search path, > update PKG_CONFIG_PATH for pkg-config to find the .pc file in non-standard > location > > The error log is puzzled because I indeed have configured PKG_CONFIG_PATH > in the right way. After checking the config.log we found libibverbs is > needed when linking with dpdk: > > configure:31517: checking whether linking with dpdk works > configure:31533: gcc -o conftest -g -O2 -DHAVE_AVX512F -mno-avx512f > -I/usr/local/include -include rte_config.h -march=native -mno-avx512f > conftest.c -Wl,--whole-archive -L/usr/local/lib64 -l:librte_common_cpt.a > -l:librte_common_dpaax.a -l:librte_common_iavf.a > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > -l:librte_bus_vmbus.a -l:librte_common_qat.a -l:librte_mempool_bucket.a > -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a > -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a > -l:librte_mempool_ring.a -l:librte_mempool_stack.a > -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a > -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a > -l:librte_net_bnx2x.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a > -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a > -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a > -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a > -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a > -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a > -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a > -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a > -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a > -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a > -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a > -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > -l:librte_compress_octeontx.a -l:librte_compress_zlib.a > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_baseband_null.a > -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a > -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a > -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a > -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a > -l:librte_stack.a -l:librte_security.a -l:librte_sched.a > -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a > -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a > -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a > -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a > -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost > -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib > -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm > -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso > -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev > -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl > -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lz -lpcap > -libverbs -ldl -lpcap -lnuma -lnuma -latomic -lpthread -lrt -lm -lunbound >> &5 > /usr/bin/ld: cannot find -libverbs Hi. I'm puzzled with how did you got into this situation in a first place? ibverbs should not be required if none of mlx drivers built into DPDK. It must be a problem with pkg-config file generated by DPDK build process. What DPDK version are you using? Does it have some custom changes? Best regards, Ilya Maximets.
Hi, Maximets, I have found this build error message in both dpdk 20.11.3 and 20.11.1. I did not make any change to dpdk. I use the following command to build dpdk: meson builddir && cd builddir && meson configure --prefix=/usr/local -Dexamples='' -Dtests=false -Denable_kmods=false -Denable_docs=false -Ddisable_drivers=event/*,crypto/*,net/mlx4,net/mlx5 && ninja && ninja install I am also puzzled because DPDK is built successfully and it does not need libibverbs. Thanks Ilya Maximets <i.maximets@ovn.org> 于2021年11月10日周三 下午7:55写道: > > On 11/10/21 05:10, Harold Huang wrote: > > When I build ovs-dpdk without mlx4/mlx5 driver, the configure log is as > > follows: > > > > checking for DPDK... yes > > checking for faulty pkg-config version... no > > checking for rte_config.h... yes > > checking whether RTE_LIBRTE_VHOST_NUMA is declared... yes > > checking for library containing get_mempolicy... -lnuma > > checking whether RTE_EAL_NUMA_AWARE_HUGEPAGES is declared... yes > > checking for library containing get_mempolicy... (cached) -lnuma > > checking whether RTE_LIBRTE_PMD_PCAP is declared... yes > > checking for library containing pcap_dump_close... -lpcap > > checking whether RTE_LIBRTE_PMD_AF_XDP is declared... no > > checking whether RTE_LIBRTE_VHOST_NUMA is declared... (cached) yes > > checking whether RTE_LIBRTE_MLX5_PMD is declared... no > > checking whether RTE_LIBRTE_MLX4_PMD is declared... no > > checking for library containing dlopen... -ldl > > checking whether linking with dpdk works... no > > configure: error: Could not find DPDK library in default search path, > > update PKG_CONFIG_PATH for pkg-config to find the .pc file in non-standard > > location > > > > The error log is puzzled because I indeed have configured PKG_CONFIG_PATH > > in the right way. After checking the config.log we found libibverbs is > > needed when linking with dpdk: > > > > configure:31517: checking whether linking with dpdk works > > configure:31533: gcc -o conftest -g -O2 -DHAVE_AVX512F -mno-avx512f > > -I/usr/local/include -include rte_config.h -march=native -mno-avx512f > > conftest.c -Wl,--whole-archive -L/usr/local/lib64 -l:librte_common_cpt.a > > -l:librte_common_dpaax.a -l:librte_common_iavf.a > > -l:librte_common_octeontx.a -l:librte_common_octeontx2.a > > -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a > > -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a > > -l:librte_bus_vmbus.a -l:librte_common_qat.a -l:librte_mempool_bucket.a > > -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a > > -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a > > -l:librte_mempool_ring.a -l:librte_mempool_stack.a > > -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a > > -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a > > -l:librte_net_bnx2x.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a > > -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a > > -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a > > -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a > > -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a > > -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a > > -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a > > -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a > > -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a > > -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a > > -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a > > -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a > > -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a > > -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a > > -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a > > -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a > > -l:librte_compress_octeontx.a -l:librte_compress_zlib.a > > -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_baseband_null.a > > -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a > > -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a > > -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a > > -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a > > -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a > > -l:librte_stack.a -l:librte_security.a -l:librte_sched.a > > -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a > > -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a > > -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a > > -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a > > -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a > > -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > > -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic > > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost > > -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib > > -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm > > -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso > > -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev > > -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl > > -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev > > -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring > > -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lz -lpcap > > -libverbs -ldl -lpcap -lnuma -lnuma -latomic -lpthread -lrt -lm -lunbound > >> &5 > > /usr/bin/ld: cannot find -libverbs > > Hi. I'm puzzled with how did you got into this situation in a first place? > ibverbs should not be required if none of mlx drivers built into DPDK. > It must be a problem with pkg-config file generated by DPDK build process. > > What DPDK version are you using? Does it have some custom changes? > > Best regards, Ilya Maximets.
On Wed, Nov 10, 2021 at 1:16 PM Harold Huang <baymaxhuang@gmail.com> wrote: > > Hi, Maximets, > > I have found this build error message in both dpdk 20.11.3 and > 20.11.1. I did not make any change to dpdk. I use the following > command to build dpdk: > > meson builddir && cd builddir && meson configure > --prefix=/usr/local -Dexamples='' -Dtests=false -Denable_kmods=false > -Denable_docs=false > -Ddisable_drivers=event/*,crypto/*,net/mlx4,net/mlx5 && ninja && > ninja install > > I am also puzzled because DPDK is built successfully and it does not > need libibverbs. The common/mlx5 driver is still enabled with this configuration. Try: -Ddisable_drivers=event/*,crypto/*,net/mlx4,*/mlx5
---------- Forwarded message --------- 发件人: Harold Huang <baymaxhuang@gmail.com> Date: 2021年11月10日周三 下午8:53 Subject: Re: [ovs-dev] [PATCH] build: check libibverbs deps before linking with dpdk To: David Marchand <david.marchand@redhat.com> Yes, I know it because it has other mlx5 drivers. But the mlx5 is indeed skipped because I do not install any mlx5 dependency, part of dpdk build log is as follows: Message: ================= Content Skipped ================= libs: drivers: common/mvep: missing dependency, "libmusdk" common/mlx5: missing dependency, "mlx5" crypto/qat: Explicitly disabled via build config net/ipn3ke: missing dependency, "libfdt" net/mlx4: explicitly disabled via build config net/mlx5: explicitly disabled via build config net/mvneta: missing dependency, "libmusdk" net/mvpp2: missing dependency, "libmusdk" net/nfb: missing dependency, "libnfb" net/szedata2: missing dependency, "libsze2" raw/ifpga: missing dependency, "libfdt" crypto/aesni_gcm: explicitly disabled via build config crypto/aesni_mb: explicitly disabled via build config crypto/armv8: explicitly disabled via build config crypto/bcmfs: explicitly disabled via build config crypto/caam_jr: explicitly disabled via build config crypto/ccp: explicitly disabled via build config crypto/dpaa_sec: explicitly disabled via build config crypto/dpaa2_sec: explicitly disabled via build config crypto/kasumi: explicitly disabled via build config crypto/mvsam: explicitly disabled via build config crypto/nitrox: explicitly disabled via build config crypto/null: explicitly disabled via build config crypto/octeontx: explicitly disabled via build config crypto/octeontx2: explicitly disabled via build config crypto/openssl: explicitly disabled via build config crypto/scheduler: explicitly disabled via build config crypto/snow3g: explicitly disabled via build config crypto/virtio: explicitly disabled via build config crypto/zuc: explicitly disabled via build config compress/isal: missing dependency, "libisal" regex/mlx5: missing internal dependency, "common_mlx5" vdpa/mlx5: missing internal dependency, "common_mlx5" event/dlb: explicitly disabled via build config event/dlb2: explicitly disabled via build config event/dpaa: explicitly disabled via build config event/dpaa2: explicitly disabled via build config event/octeontx2: explicitly disabled via build config event/opdl: explicitly disabled via build config event/skeleton: explicitly disabled via build config event/sw: explicitly disabled via build config event/dsw: explicitly disabled via build config event/octeontx: explicitly disabled via build config This dpdk.pc file is as follows: prefix=/usr/local libdir=${prefix}/lib64 includedir=${prefix}/include Name: DPDK Description: The Data Plane Development Kit (DPDK). Note that CFLAGS might contain an -march flag higher than typical baseline. This is required for a number of static inline functions in the public headers. Version: 20.11.3 Requires: libdpdk-libs Requires.private: libbpf, zlib, libpcap, libelf Libs.private: -Wl,--whole-archive -L${libdir} -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_af_xdp.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnx2x.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_compress_octeontx.a -l:librte_compress_zlib.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic Cflags: -I${includedir} The libdpdk-libs.pc is as follows: prefix=/usr/local libdir=${prefix}/lib64 includedir=${prefix}/include Name: dpdk-libs Description: Internal-only DPDK pkgconfig file. Not for direct use. Use libdpdk.pc instead of this file to query DPDK compile/link arguments Version: 20.11.3 Libs: -Wl,--as-needed -L${libdir} -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs Libs.private: -pthread -lm -ldl -lnuma -lpcap Cflags: -I${includedir} -I${includedir} -include rte_config.h -march=native -mno-avx512f FIY, both dpdk and ovs are built in CentOS Linux release 8.2.2004 with a native kernel 4.18.0-193.el8.x86_64. David Marchand <david.marchand@redhat.com> 于2021年11月10日周三 下午8:36写道: > > On Wed, Nov 10, 2021 at 1:16 PM Harold Huang <baymaxhuang@gmail.com> wrote: > > > > Hi, Maximets, > > > > I have found this build error message in both dpdk 20.11.3 and > > 20.11.1. I did not make any change to dpdk. I use the following > > command to build dpdk: > > > > meson builddir && cd builddir && meson configure > > --prefix=/usr/local -Dexamples='' -Dtests=false -Denable_kmods=false > > -Denable_docs=false > > -Ddisable_drivers=event/*,crypto/*,net/mlx4,net/mlx5 && ninja && > > ninja install > > > > I am also puzzled because DPDK is built successfully and it does not > > need libibverbs. > > The common/mlx5 driver is still enabled with this configuration. > Try: -Ddisable_drivers=event/*,crypto/*,net/mlx4,*/mlx5 > > -- > David Marchand >
On Thu, Nov 11, 2021 at 2:55 AM Harold Huang <baymaxhuang@gmail.com> wrote: > > FIY, both dpdk and ovs are built in CentOS Linux release 8.2.2004 > with a native kernel 4.18.0-193.el8.x86_64. I started a Centos 8.2 vm. No libibverbs installed by default. I compiled DPDK v20.11.3 and installed it in builddir/install The output you provided on .pc files matches what I see. No reference to libibverbs in installed DPDK: [dpdk@centos8 dpdk]$ grep -r ibverbs builddir/install [dpdk@centos8 dpdk]$ I have no problem building and linking OVS (current master branch). Here is what I see in my working OVS environment (I mean in the directory where I have the OVS sources, and where I ran sh boot.sh + PKG_CONFIG_PATH=... ./configure --with-dpdk): [dpdk@centos8 ovs]$ grep -r ibverbs . ./acinclude.m4: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./acinclude.m4: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.0: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.0: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.0:m4trace:configure.ac:197: -1- OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.0:m4trace:configure.ac:197: -1- OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/output.0:for ac_lib in '' ibverbs; do ./autom4te.cache/output.0: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./autom4te.cache/output.0:for ac_lib in '' ibverbs; do ./autom4te.cache/output.0: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./autom4te.cache/output.1:for ac_lib in '' ibverbs; do ./autom4te.cache/output.1: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./autom4te.cache/output.1:for ac_lib in '' ibverbs; do ./autom4te.cache/output.1: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./autom4te.cache/traces.2: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.2: OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.2:m4trace:configure.ac:197: -1- OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/traces.2:m4trace:configure.ac:197: -1- OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ./autom4te.cache/output.2:for ac_lib in '' ibverbs; do ./autom4te.cache/output.2: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./autom4te.cache/output.2:for ac_lib in '' ibverbs; do ./autom4te.cache/output.2: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./configure:for ac_lib in '' ibverbs; do ./configure: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 ./configure:for ac_lib in '' ibverbs; do ./configure: as_fn_error $? "unable to find libibverbs, install the dependency package" "$LINENO" 5 OVS requests ibverbs only if mlx drivers are present in DPDK build (+ option RTE_IBVERBS_LINK_DLOPEN iirc). For your issue, things that come to mind are an outdated build directory, maybe following a pull from OVS sources, or maybe you had ibverbs installed in the past on this system, and now it has been removed. Did you compile OVS from a scratch/clean environment?
David Marchand <david.marchand@redhat.com> 于2021年11月12日周五 下午7:01写道: > > OVS requests ibverbs only if mlx drivers are present in DPDK build (+ > option RTE_IBVERBS_LINK_DLOPEN iirc). > > For your issue, things that come to mind are an outdated build > directory, maybe following a pull from OVS sources, or maybe you had > ibverbs installed in the past on this system, and now it has been > removed. > > Did you compile OVS from a scratch/clean environment? Hi, David, I also started a clean centos 8.2 vm environment and init the vm environment with the following command: $dnf -y install 'dnf-command(config-manager)' && dnf config-manager --set-enabled PowerTools && dnf install -y gcc make numactl-devel meson autoconf automake libtool unbound unbound-devel I could also build dpdk and link ovs successfully. But after I installed libpcap-devel and rebuild dpdk, I could not link ovs anymore. And the error message is the same as what I posted before. I found libpcap.pc also has a dependency about libibverbs: # # pkg-config file for libpcap. # # These variables come from the configure script, so includedir and # libdir may be defined in terms of prefix and exec_prefix, so the # latter must be defined as well. # prefix="/usr" exec_prefix="/usr" includedir="/usr/include" libdir="/usr/lib64" Name: libpcap Description: Platform-independent network traffic capture library Version: 1.9.1 Libs: -L${libdir} -lpcap Libs.private: -libverbs Cflags: -I${includedir} As a result, maybe libverbs should also be added in: AC_CHECK_DECL([RTE_LIBRTE_PMD_PCAP], [ OVS_FIND_DEPENDENCY([pcap_dump_close], [pcap], [libpcap],[libverbs]) ], [], [[#include <rte_config.h>]]) Thanks Harold Huang
Hello, On Sun, Nov 14, 2021 at 10:27 AM Harold Huang <baymaxhuang@gmail.com> wrote: > David Marchand <david.marchand@redhat.com> 于2021年11月12日周五 下午7:01写道: > > OVS requests ibverbs only if mlx drivers are present in DPDK build (+ > > option RTE_IBVERBS_LINK_DLOPEN iirc). > > > > For your issue, things that come to mind are an outdated build > > directory, maybe following a pull from OVS sources, or maybe you had > > ibverbs installed in the past on this system, and now it has been > > removed. > > > > Did you compile OVS from a scratch/clean environment? > > Hi, David, > I also started a clean centos 8.2 vm environment and init the vm > environment with the following command: > $dnf -y install 'dnf-command(config-manager)' && dnf config-manager > --set-enabled PowerTools && dnf install -y gcc make numactl-devel > meson autoconf automake libtool unbound unbound-devel > > I could also build dpdk and link ovs successfully. But after I > installed libpcap-devel and rebuild dpdk, I could not link ovs > anymore. And the error message is the same as what I posted before. I > found libpcap.pc also has a dependency about libibverbs: > # > # pkg-config file for libpcap. > # > # These variables come from the configure script, so includedir and > # libdir may be defined in terms of prefix and exec_prefix, so the > # latter must be defined as well. > # > prefix="/usr" > exec_prefix="/usr" > includedir="/usr/include" > libdir="/usr/lib64" > > Name: libpcap > Description: Platform-independent network traffic capture library > Version: 1.9.1 > Libs: -L${libdir} -lpcap > Libs.private: -libverbs > Cflags: -I${includedir} I can reproduce the issue now. As you describe, I installed libpcap-devel, rebuilt and installed DPDK, then: [dpdk@centos8 build]$ PKG_CONFIG_PATH=/home/dpdk/dpdk/builddir/install/lib64/pkgconfig ../configure --with-dpdk ... checking for struct tcf_t.firstuse... yes checking whether dpdk is enabled... yes checking for DPDK... yes checking for faulty pkg-config version... no checking for rte_config.h... yes checking whether RTE_LIBRTE_VHOST_NUMA is declared... yes checking for library containing get_mempolicy... -lnuma checking whether RTE_EAL_NUMA_AWARE_HUGEPAGES is declared... yes checking for library containing get_mempolicy... (cached) -lnuma checking whether RTE_NET_PCAP is declared... yes checking for library containing pcap_dump_close... -lpcap checking whether RTE_NET_AF_XDP is declared... no checking whether RTE_LIBRTE_VHOST_NUMA is declared... (cached) yes checking whether RTE_NET_MLX5 is declared... no checking whether RTE_NET_MLX4 is declared... no checking for library containing dlopen... -ldl checking whether linking with dpdk works... no configure: error: Could not find DPDK library in default search path, update PKG_CONFIG_PATH for pkg-config to find the .pc file in non-standard location (fwiw, linking against dpdk dynamically works). The problem is either on dpdk side or (I would lean to) Centos packaging, not OVS. It can be reproduced with DPDK examples: [dpdk@centos8 build]$ cd ~/dpdk/builddir/install/share/dpdk/examples/helloworld/ [dpdk@centos8 helloworld]$ PKG_CONFIG_PATH=/home/dpdk/dpdk/builddir/install/lib64/pkgconfig make static cc -O3 -I/home/dpdk/dpdk/builddir/install/include -include rte_config.h -march=native -mno-avx512f -DALLOW_EXPERIMENTAL_API main.c -o build/helloworld-static -Wl,--whole-archive -L/home/dpdk/dpdk/builddir/install/lib64 -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnx2x.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_compress_octeontx.a -l:librte_compress_zlib.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lz -lpcap -libverbs /usr/bin/ld: cannot find -libverbs collect2: error: ld returned 1 exit status make: *** [Makefile:35: build/helloworld-static] Error 1 [dpdk@centos8 helloworld]$ ls -1 /usr/lib64/libpcap.so* /usr/lib64/libibverbs.so* /usr/lib64/libibverbs.so.1 /usr/lib64/libibverbs.so.1.11.32.0 /usr/lib64/libpcap.so /usr/lib64/libpcap.so.1 /usr/lib64/libpcap.so.1.9.1 Absence of libibverbs.so is explained by the the absence of libibverbs-devel package. [dpdk@centos8 helloworld]$ rpm -q libpcap libpcap-devel libibverbs libibverbs-devel libpcap-1.9.1-5.el8.x86_64 libpcap-devel-1.9.1-5.el8.x86_64 libibverbs-32.0-4.el8.x86_64 package libibverbs-devel is not installed Problem is solved for me, by installing libibverbs-devel. Copying Bruce and Timothy who may have an opinion on this.
diff --git a/acinclude.m4 b/acinclude.m4 index 8ab690f47..73a8861d6 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -461,17 +461,17 @@ AC_DEFUN([OVS_CHECK_DPDK], [ AC_CHECK_DECL([RTE_NET_MLX5], [dnl found AC_CHECK_DECL([RTE_IBVERBS_LINK_DLOPEN], [], [dnl not found OVS_FIND_DEPENDENCY([mlx5dv_create_wq], [mlx5], [libmlx5]) - OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ], [[#include <rte_config.h>]]) ], [], [[#include <rte_config.h>]]) AC_CHECK_DECL([RTE_NET_MLX4], [dnl found AC_CHECK_DECL([RTE_IBVERBS_LINK_DLOPEN], [], [dnl not found OVS_FIND_DEPENDENCY([mlx4dv_init_obj], [mlx4], [libmlx4]) - OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) ], [[#include <rte_config.h>]]) ], [], [[#include <rte_config.h>]]) + OVS_FIND_DEPENDENCY([verbs_init_cq], [ibverbs], [libibverbs]) + # DPDK uses dlopen to load plugins. OVS_FIND_DEPENDENCY([dlopen], [dl], [libdl])