HELP: DPDK with ConnectX-3 problem

Infrastructure & Networking Software And Drivers Application Accelerator Software
1

Hi, all

I’m trying to use dpdk with Mellanox ConnectX-3.

But, mlnx4 pmd cannot work fine.

Following is my environment:

OS: CentOS 6.5 (x86_64) with kernel version 2.6.32-431.29.2.el6.x86_64

Mellanox OFED: MLNX_OFED_LINUX-3.1-1.1.0.1-rhel6.5-x86_64

DPDK: MLNX_DPDK-2.1_1.1

Device: ConnectX®-3 EN network interface card with firmware version 2.35.5100

I’m trying to run testpmd, but librte_pmd_mlx4 reports “ibv_exp_query_device() failed”,

and EAL reports “No probed ethernet devices”.

What can I do next to work mlx4 pmd fine?

Following is the testpmd’s output:

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 1 on socket 0

EAL: Detected lcore 2 as core 2 on socket 0

EAL: Detected lcore 3 as core 3 on socket 0

EAL: Detected lcore 4 as core 4 on socket 0

EAL: Detected lcore 5 as core 5 on socket 0

EAL: Detected lcore 6 as core 8 on socket 0

EAL: Detected lcore 7 as core 9 on socket 0

EAL: Detected lcore 8 as core 10 on socket 0

EAL: Detected lcore 9 as core 11 on socket 0

EAL: Detected lcore 10 as core 12 on socket 0

EAL: Detected lcore 11 as core 13 on socket 0

EAL: Detected lcore 12 as core 0 on socket 1

EAL: Detected lcore 13 as core 1 on socket 1

EAL: Detected lcore 14 as core 2 on socket 1

EAL: Detected lcore 15 as core 3 on socket 1

EAL: Detected lcore 16 as core 4 on socket 1

EAL: Detected lcore 17 as core 5 on socket 1

EAL: Detected lcore 18 as core 8 on socket 1

EAL: Detected lcore 19 as core 9 on socket 1

EAL: Detected lcore 20 as core 10 on socket 1

EAL: Detected lcore 21 as core 11 on socket 1

EAL: Detected lcore 22 as core 12 on socket 1

EAL: Detected lcore 23 as core 13 on socket 1

EAL: Support maximum 128 logical core(s) by configuration.

EAL: Detected 24 lcore(s)

EAL: Setting up physically contiguous memory…

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x7f2940000000 (size = 0x80000000)

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x7f2880000000 (size = 0x80000000)

EAL: Requesting 2 pages of size 1024MB from socket 0

EAL: Requesting 2 pages of size 1024MB from socket 1

EAL: TSC frequency is ~2300002 KHz

EAL: Master lcore 0 is ready (tid=f6cdc8a0;cpuset=[0])

EAL: lcore 2 is ready (tid=f58d2700;cpuset=[2])

EAL: lcore 1 is ready (tid=f62d3700;cpuset=[1])

EAL: lcore 3 is ready (tid=f4ed1700;cpuset=[3])

EAL: PCI device 0000:02:00.0 on NUMA socket 0

EAL: probe driver: 15b3:1003 librte_pmd_mlx4

PMD: librte_pmd_mlx4: PCI information matches, using device “mlx4_0” (VF: false)

PMD: librte_pmd_mlx4: 1 port(s) detected

PMD: librte_pmd_mlx4: ibv_exp_query_device() failed

EAL: PCI device 0000:03:00.0 on NUMA socket 0

EAL: probe driver: 8086:1528 rte_ixgbe_pmd

EAL: Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:81:00.0 on NUMA socket 1

EAL: probe driver: 8086:1521 rte_igb_pmd

EAL: Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:81:00.1 on NUMA socket 1

EAL: probe driver: 8086:1521 rte_igb_pmd

EAL: Not managed by a supported kernel driver, skipped

EAL: No probed ethernet devices

Best regards,

Masaru Kawashima

2

You should not remove this ibv_exp_query_device(), this should work (and working in identical setup in my lab) with no issues. There is another problem causing this.

  1. Are you sure you have only one version of libibverbs in the system?

  2. Can you please run some standard commands like ibstat, ibdev2netdev and ibv_devinfo (and paste it here) just to make sure?

3

Could you check if it happens with the MOFED v3.2?

http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers

4

DPDK2.2.0, SLES12SP1 (3.12.49-11-default) and OFED v3.2 works on my setup with Mellanox ConnectX-3 Pro.

5

I found ibv_exp_query_device()'s return code is ENOSYS (= 38: Function not implemented) !?

Can I remove this ibv_exp_query_device() call from mlx4 pmd?