Host configuration:
OS: Debian GNU/Linux 10 (buster)
Kernel: 4.19.0-17-amd64
Architecture: x86_64
mlnx_oefd version: MLNX_OFED_LINUX-5.4-1.0.3.0-debian10.8-x86_64
Guest configuration:
OS: Ubuntu 16.04.1 LTS
Kernel: 4.4.0-47-generic
mlnx_ofed version: MLNX_OFED_LINUX-5.4-1.0.3.0-ubuntu16.04-x86_64
Dpdk version: 18.11
Please see the attachment for more detailed configuration.
This is what I get when I try to run testpmd on the guest.
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 15b3:101a net_mlx5
net_mlx5: mlx5_flow.c:364: mlx5_flow_discover_priorities(): port 0 verbs maximum priority: 8 expected 8/16
testpmd: /home/sbandi/dpdk/drivers/net/mlx5/mlx5.c:1259: mlx5_dev_spawn: Assertion `(mlx5_glue->dealloc_pd(pd)) == 0’ failed.
Please let me know what is going wrong here.
mlnx.txt (25.9 KB)
mlx5_flow_discover_priorities is a verbs based function and Mellanox OFED v5.4 more is rdma-core based. There were changes in this portion of the code
Check DPDK v18.11 release notes to find tested MOFED version
http://doc.dpdk.org/guides-18.11/rel_notes/release_18_11.html
I would suggest to switch to latest LTS DPDK and see if the issue is still there. It is also worth trying latest DPDK version.
In addition, firmware version running on the host is not supported by this version of the MOFED. Please, check MOFED release notes for supported version. As you are running OEM (DELL) card, you might contact OEM to get relevant version. If you have a Mellanox branded PSID, you should be able to get latest version from Mellanox web site.
You might also try MOFED v4.9 LTS, that still using verbs. Looks like your OS’es are supported - https://docs.mellanox.com/display/MLNXOFEDv494080/General+Support+in+MLNX_OFED
Thank you for the response.
I did try downgrading to MOFED v4.9 LTS, but now I’m facing an issue with loading libibverbs library while running testpmd.
root@cep:~# ./testpmd -w 0000:00:03.0
net_mlx5: mlx5.c:1712: mlx5_glue_init(): cannot load glue library: /usr/lib/libibverbs.so.1: version `IBVERBS_1.8’ not found (required by /usr/lib/x86_64-linux-gnu/librte_pmd_mlx5_glue.so.18.11.0)
net_mlx5: mlx5.c:1730: mlx5_glue_init(): cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
The libibverbds is part of the MOFED installation, so I’m not sure what dpdk is complaining about. Would you have any idea?
Seems like MOFED installation issue. MOFED does contain libibverbs.
I would highly recommend to use latest 5.4 and 20.11 DPDK version first. Install MOFED with ‘mlnxofedinstall --all’ and the compile DPDK
Download, extract and run
meson --prefix= build
Using prefix is very recommended until you finish our development cycle.
ninja -C build install
It could be that you have to add meson/ninja packages to your OS