Unable to use Mellanox CX5 VF in my DPDK application

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hi all,
I have a VM running on top of VMWare ESXi which gets a Mellanox VF exposed.
This is the output of dpdk-devbind -s inside the VM, after installing the OFED v5.4:

[root@localhost MLNX_OFED_LINUX-5.4-1.0.3.0-rhel7.9-x86_64]# dpdk-devbind -s

Network devices using DPDK-compatible driver
============================================
0000:04:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3,vfio-pci,uio_pci_generic
0000:13:00.0 'Ethernet Virtual Function 700 Series 154c' drv=igb_uio unused=iavf,vfio-pci,uio_pci_generic

Network devices using kernel driver
===================================
0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens192 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
0000:1b:00.0 'MT28800 Family [ConnectX-5 Ex Virtual Function] 101a' if=ens256 drv=mlx5_core unused=igb_uio,vfio-pci,uio_pci_generic

...

However when I try to start my DPDK application on it I get:

DPDK: EAL argv param is [hmcapture --syslog=local0 --lcores=0@0,1@1,2@2,3@3,4@1 --pci-whitelist=0000:04:00.0 --pci-whitelist=0000:13:00.0 --pci-whitelist=0000:1b:00.0]
EAL: Detected 12 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: Device 0000:04:00.0 is not NUMA-aware, defaulting socket to 0
EAL: probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:13:00.0 on NUMA socket -1
EAL: Device 0000:13:00.0 is not NUMA-aware, defaulting socket to 0
EAL: probe driver: 8086:154c net_i40e_vf
EAL: PCI device 0000:1b:00.0 on NUMA socket -1
EAL: Device 0000:1b:00.0 is not NUMA-aware, defaulting socket to 0
EAL: probe driver: 15b3:101a net_mlx5
net_mlx5: unable to recognize master/representors on the multiple IB devices
EAL: Requested device 0000:1b:00.0 cannot be used

I’m trying to understand this “net_mlx5: unable to recognize master/representors on the multiple IB devices” but I cannot fix it… any suggestion?

thanks a lot for any hint

2

Replying to myself: it turns out that my application was running inside a container having its own network namespace, where the network interface device (e.g. “enp1s0f0”) was not visible.
What I discovered is that even when using DPDK and PCI addresses for NIC port selection, apparently Mellanox mlx5_core driver still needs to have visibility on the network interface (e.g. “enp1s0f0”). By using the host-device plugin (GitHub - containernetworking/plugins: Some reference and example networking plugins, maintained by the CNI team.) the network interface has been made visible inside the container (a Kubernetes POD) and that fixed this error message from DPDK.

3

Hi Francesco,

Thank you for updating the forum on the fix details.

Thanks,
Namrata.