dpdk and connectx3 problem

st.uzver · November 25, 2015, 1:40pm

hello, I got compiled dpdk-2.1-1.1 from mellanox website

I got mlnx ofed 3.1 1.0.3

I trying to start testpmd and got errors

I using centos 7.1.

./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n4

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 1 on socket 0

EAL: Detected lcore 2 as core 2 on socket 0

EAL: Detected lcore 3 as core 3 on socket 0

EAL: Detected lcore 4 as core 4 on socket 0

EAL: Detected lcore 5 as core 5 on socket 0

EAL: Support maximum 128 logical core(s) by configuration.

EAL: Detected 6 lcore(s)

EAL: VFIO modules not all loaded, skip VFIO support…

EAL: Setting up physically contiguous memory…

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7ff0c1200000 (size = 0x200000)

EAL: Ask a virtual area of 0x27c00000 bytes

EAL: Virtual area found at 0x7ff099400000 (size = 0x27c00000)

EAL: Ask a virtual area of 0x1000000 bytes

EAL: Virtual area found at 0x7ff098200000 (size = 0x1000000)

EAL: Ask a virtual area of 0x3e000000 bytes

EAL: Virtual area found at 0x7ff05a000000 (size = 0x3e000000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7ff059c00000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7ff059800000 (size = 0x200000)

EAL: Ask a virtual area of 0xf400000 bytes

EAL: Virtual area found at 0x7ff04a200000 (size = 0xf400000)

EAL: Ask a virtual area of 0x79800000 bytes

EAL: Virtual area found at 0x7fefd0800000 (size = 0x79800000)

EAL: Ask a virtual area of 0x10000000 bytes

EAL: Virtual area found at 0x7fefc0600000 (size = 0x10000000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fefc0200000 (size = 0x200000)

EAL: Requesting 2048 pages of size 2MB from socket 0

EAL: TSC frequency is ~3699897 KHz

EAL: Master lcore 0 is ready (tid=c314f900;cpuset=[0])

EAL: lcore 1 is ready (tid=bf9e7700;cpuset=[1])

EAL: lcore 3 is ready (tid=be9e5700;cpuset=[3])

EAL: lcore 2 is ready (tid=bf1e6700;cpuset=[2])

EAL: PCI device 0000:01:00.0 on NUMA socket 0

EAL: probe driver: 15b3:1003 librte_pmd_mlx4

PMD: librte_pmd_mlx4: PCI information matches, using device “mlx4_0” (VF: false)

PMD: librte_pmd_mlx4: 2 port(s) detected

PMD: librte_pmd_mlx4: port 1 MAC address is f4:52:14:22:a2:71

PMD: librte_pmd_mlx4: port 2 MAC address is f4:52:14:22:a2:72

EAL: PCI device 0000:0a:00.0 on NUMA socket -1

EAL: probe driver: 8086:10d3 rte_em_pmd

EAL: Not managed by a supported kernel driver, skipped

Configuring Port 0 (socket 0)

PMD: librte_pmd_mlx4: 0x8e00e0: TX queues number update: 0 → 1

PMD: librte_pmd_mlx4: 0x8e00e0: RX queues number update: 0 → 1

PMD: librte_pmd_mlx4: 0x8e00e0: QP creation failure: Invalid argument

Fail to configure port 0 tx queues

EAL: Error - exiting with code: 1

Cause: Start ports failed

yairi · November 30, 2015, 1:01pm

Hi Alexander,

can you try installing the OFED and running with none real-time kernel?

the Mellanox OFED driver currently don’t have support for RT kernels.

st.uzver · November 30, 2015, 1:29pm

but with no dpdk it works with RT

st.uzver · November 30, 2015, 1:29pm

I tryid it with native kernet on this server. and it got the same problem.

but I got tryied on clean centos install with native kernel with ofed with dpdk and its work…

can one of this kernel options make the problem with dpdk ?

ipv6.disable=1 net.ifnames=0 nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce idle=poll audit=0 nmi_watchdog=0 nowatchdog

rdarbha · November 29, 2015, 11:29am

Hi,

Please make sure you have follow the instructions in MLNX_DPDK QSG.

After OFED installation, configure options mlx4_core log_num_mgm_entry_size=-7 , run /etc/init.d/openibd restart

Compile DPDK (remove build dir before)

If you have old FW installed on you NIC, make sure new FW was installed during OFED installation - in this case you need to restart you server

Thanks

Olga

st.uzver · November 30, 2015, 7:14am

we got updated FW.

we got option

options mlx4_core log_num_mgm_entry_size=-1

cause we using vlans.

and we got errors.

st.uzver · November 30, 2015, 7:26am

I tried compile it with -7. the same result:

./build/build/app/test-pmd/testpmd -c 0xf -n 4 -w 0000:01:00.0 – --rxq=2 --txq=2 -i

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 1 on socket 0

EAL: Detected lcore 2 as core 2 on socket 0

EAL: Detected lcore 3 as core 3 on socket 0

EAL: Detected lcore 4 as core 4 on socket 0

EAL: Detected lcore 5 as core 5 on socket 0

EAL: Support maximum 128 logical core(s) by configuration.

EAL: Detected 6 lcore(s)

EAL: VFIO modules not all loaded, skip VFIO support…

EAL: Setting up physically contiguous memory…

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7caaa00000 (size = 0x200000)

EAL: Ask a virtual area of 0x28c00000 bytes

EAL: Virtual area found at 0x7f7c81c00000 (size = 0x28c00000)

EAL: Ask a virtual area of 0x1000000 bytes

EAL: Virtual area found at 0x7f7c80a00000 (size = 0x1000000)

EAL: Ask a virtual area of 0x40000000 bytes

EAL: Virtual area found at 0x7f7c40800000 (size = 0x40000000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c40400000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c40000000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c3fc00000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c3f800000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c3f400000 (size = 0x200000)

EAL: Ask a virtual area of 0x1400000 bytes

EAL: Virtual area found at 0x7f7c3de00000 (size = 0x1400000)

EAL: Ask a virtual area of 0xa00000 bytes

EAL: Virtual area found at 0x7f7c3d200000 (size = 0xa00000)

EAL: Ask a virtual area of 0xe00000 bytes

EAL: Virtual area found at 0x7f7c3c200000 (size = 0xe00000)

EAL: Ask a virtual area of 0x1600000 bytes

EAL: Virtual area found at 0x7f7c3aa00000 (size = 0x1600000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c3a600000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7c3a200000 (size = 0x200000)

EAL: Ask a virtual area of 0xf800000 bytes

EAL: Virtual area found at 0x7f7c2a800000 (size = 0xf800000)

EAL: Ask a virtual area of 0x69000000 bytes

EAL: Virtual area found at 0x7f7bc1600000 (size = 0x69000000)

EAL: Ask a virtual area of 0x800000 bytes

EAL: Virtual area found at 0x7f7bc0c00000 (size = 0x800000)

EAL: Ask a virtual area of 0x400000 bytes

EAL: Virtual area found at 0x7f7bc0600000 (size = 0x400000)

EAL: Ask a virtual area of 0x800000 bytes

EAL: Virtual area found at 0x7f7bbfc00000 (size = 0x800000)

EAL: Ask a virtual area of 0x400000 bytes

EAL: Virtual area found at 0x7f7bbf600000 (size = 0x400000)

EAL: Ask a virtual area of 0x2800000 bytes

EAL: Virtual area found at 0x7f7bbcc00000 (size = 0x2800000)

EAL: Ask a virtual area of 0x800000 bytes

EAL: Virtual area found at 0x7f7bbc200000 (size = 0x800000)

EAL: Ask a virtual area of 0x4000000 bytes

EAL: Virtual area found at 0x7f7bb8000000 (size = 0x4000000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7bb7c00000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7bb7800000 (size = 0x200000)

EAL: Ask a virtual area of 0xfc00000 bytes

EAL: Virtual area found at 0x7f7ba7a00000 (size = 0xfc00000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f7ba7600000 (size = 0x200000)

EAL: Requesting 2048 pages of size 2MB from socket 0

EAL: TSC frequency is ~3699895 KHz

EAL: Master lcore 0 is ready (tid=ac7db900;cpuset=[0])

EAL: lcore 1 is ready (tid=a6de7700;cpuset=[1])

EAL: lcore 2 is ready (tid=a65e6700;cpuset=[2])

EAL: lcore 3 is ready (tid=a5de5700;cpuset=[3])

EAL: PCI device 0000:01:00.0 on NUMA socket 0

EAL: probe driver: 15b3:1003 librte_pmd_mlx4

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4769: mlx4_pci_devinit(): using driver device index 0

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4791: mlx4_pci_devinit(): checking device “mlx4_0”

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4802: mlx4_pci_devinit(): PCI information matches, using device “mlx4_0” (VF: false)

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4822: mlx4_pci_devinit(): device opened

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4825: mlx4_pci_devinit(): 2 port(s) detected

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4847: mlx4_pci_devinit(): using port 1 (00000001)

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4915: mlx4_pci_devinit(): device flags: IBV_DEVICE_QPG IBV_DEVICE_RSS

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4918: mlx4_pci_devinit(): maximum RSS indirection table size: 256

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4927: mlx4_pci_devinit(): checksum offloading is not supported

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4932: mlx4_pci_devinit(): L2 tunnel checksum offloads are not supported

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4973: mlx4_pci_devinit(): port 1 MAC address is f4:52:14:22:a2:71

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4987: mlx4_pci_devinit(): port 1 ifname is “mlnx0”

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4994: mlx4_pci_devinit(): port 1 MTU is 1500

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:5021: mlx4_pci_devinit(): forcing Ethernet interface up

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4847: mlx4_pci_devinit(): using port 2 (00000002)

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4862: mlx4_pci_devinit(): bad state for port 2: “down” (1)

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4915: mlx4_pci_devinit(): device flags: IBV_DEVICE_QPG IBV_DEVICE_RSS

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4918: mlx4_pci_devinit(): maximum RSS indirection table size: 256

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4927: mlx4_pci_devinit(): checksum offloading is not supported

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4932: mlx4_pci_devinit(): L2 tunnel checksum offloads are not supported

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4973: mlx4_pci_devinit(): port 2 MAC address is f4:52:14:22:a2:72

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4987: mlx4_pci_devinit(): port 2 ifname is “mlnx1”

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4994: mlx4_pci_devinit(): port 2 MTU is 1500

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:5021: mlx4_pci_devinit(): forcing Ethernet interface up

Interactive-mode selected

Configuring Port 0 (socket 0)

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:690: dev_configure(): 0x8c3560: TX queues number update: 0 → 2

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:696: dev_configure(): 0x8c3560: RX queues number update: 0 → 2

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:3399: rxq_setup(): priv->device_attr.max_qp_wr is 16351

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:3401: rxq_setup(): priv->device_attr.max_sge is 32

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:3066: rxq_setup_qp_rss(): initializing parent RSS queue

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:3412: rxq_setup(): 0x8c3560: QP creation failure: Invalid argument

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:2440: rxq_cleanup(): cleaning up 0x7ffdc3e1cc40

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:1939: rxq_free_elts(): 0x7ffdc3e1cc40: freeing WRs

Fail to configure port 0

EAL: Error - exiting with code: 1

Cause: Start ports failed

rdarbha · November 30, 2015, 7:32am

Please send the following:

dmesg output after the failure
ofed_info -s
ibv_devinfo

What is strange in your case is:

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4927: mlx4_pci_devinit(): checksum offloading is not supported

/root/MLNX_DPDK-2.1_1.1/drivers/net/mlx4/mlx4.c:4932: mlx4_pci_devinit(): L2 tunnel checksum offloads are not supported

It is not related to the error but it indicates that something is wrong with the installation/setup - since this should be supported with the versions you are currently working

yairi · December 6, 2018, 11:19am

Original message from @Alex Vasin

Topic		Replies	Views
DPDK-OVS using connectx3-Pro binding issues Application Accelerator Software ethtool , test_pmd	10	1039	January 26, 2016
DPDK not working with ConnectX-3 card on Openstack virtual setup Software And Drivers openstack , port	8	1192	May 10, 2019
HELP: DPDK with ConnectX-3 problem Application Accelerator Software	4	677	March 5, 2016
Unable to build OFED from sources for Linux kernel 4.14 Mellanox OFED	1	342	January 29, 2018
DPDK testpmd cannot attach flow rules Application Accelerator Software	1	1163	January 20, 2018
DPDK APIs rte_eth_dev_count_avail() returns 0 with the MLX5 NIC card Software And Drivers mst	8	2228	November 29, 2019
DPDK ports not getting detected on CX-5 EX.. What did i miss here..? net_mlx5: mlx5.c:1740: mlx5_glue_init(): cannot load glue library: librte_pmd_mlx5_glue.so.18.11.0: cannot open shared object file: No such file or directory Software And Drivers	4	957	June 30, 2020
ConnectX-6 OVS DPDK ERROR Ethernet Adapter Cards ubuntu	3	790	February 13, 2024
Trouble running dpdk-testpmd on a KVM machine using ConnectX-5 interfaces Software And Drivers kernel , ubuntu , dpdk , adapters-and-cables	3	1856	November 12, 2021
How to install ofed sw for dpdk Mellanox OFED	9	1118	June 10, 2018

dpdk and connectx3 problem

Related topics