Is it possible to execute latest DPDK in rdma?

I want to load 19.05.0 dpdk with latest rdma in CentOS7.2.

I tried to load it, but it failed.

Is it possible to start dpdk using rdma, not OFED?

I think that rdma have bugs.

lspci info:

[root@SNIPER app]# lspci |grep Mella

3b:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]

af:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]

ifconfig info:​

※ In using OFED, mellanox interface loaded in eth2 and eth3, but now it didn’t load.

[root@SNIPER app]# ifconfig -a

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 10.0.11.64 netmask 255.255.255.0 broadcast 10.0.11.255

inet6 fe80::20d:48ff:fe4e:8c1b prefixlen 64 scopeid 0x20

ether 00:0d:48:4e:8c:1b txqueuelen 1000 (Ethernet)

RX packets 234013 bytes 23926333 (22.8 MiB)

RX errors 0 dropped 10 overruns 0 frame 0

TX packets 6534 bytes 1410452 (1.3 MiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

device memory 0x9c500000-9c5fffff

eth1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500

ether 00:0d:48:4e:8c:1c txqueuelen 1000 (Ethernet)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

device memory 0x9c200000-9c2fffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10

loop txqueuelen 0 (Local Loopback)

RX packets 324 bytes 18144 (17.7 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 324 bytes 18144 (17.7 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

rdma library version info:

[root@SNIPER app]# rpm -qa |grep rdma

rdma-core-devel-17.2-3.el7.x86_64

rdma-core-17.2-3.el7.x86_64

librdmacm-17.2-3.el7.x86_64

lsmod info:

[root@SNIPER app]# lsmod

Module Size Used by

igb_uio 13476 0

xprtrdma 50777 0

sunrpc 300421 1 xprtrdma

ib_isert 55331 0

iscsi_target_mod 295398 1 ib_isert

ib_iser 52048 0

libiscsi 57233 1 ib_iser

scsi_transport_iscsi 99909 2 ib_iser,libiscsi

ib_srpt 52416 0

target_core_mod 371914 3 iscsi_target_mod,ib_srpt,ib_isert

ib_srp 47430 0

scsi_transport_srp 20725 1 ib_srp

scsi_tgt 20027 1 scsi_transport_srp

ib_ipoib 91475 0

rdma_ucm 22569 0

ib_ucm 22546 0

ib_uverbs 47183 2 ib_ucm,rdma_ucm

ib_umad 22129 0

rdma_cm 43697 4 xprtrdma,ib_iser,rdma_ucm,ib_isert

ib_cm 46821 5 rdma_cm,ib_srp,ib_ucm,ib_srpt,ib_ipoib

ib_sa 33950 5 rdma_cm,ib_cm,ib_srp,rdma_ucm,ib_ipoib

ib_mad 47817 4 ib_cm,ib_sa,ib_srpt,ib_umad

iw_cm 41842 1 rdma_cm

ib_core 98787 14 rdma_cm,ib_cm,ib_sa,iw_cm,xprtrdma,ib_mad,ib_srp,ib_ucm,ib_iser,ib_srpt,ib_umad,ib_uverbs,ib_ipoib,ib_isert

ib_addr 18923 3 rdma_cm,ib_core,rdma_ucm

sn_snxcpu 12946 0

vtsspp 390792 0

sep5 856884 0

socperf3 582078 1 sep5

pax 13820 0

iptable_filter 12810 1

ipmi_devintf 17572 0

ipmi_msghandler 46609 1 ipmi_devintf

ip_tables 27240 1 iptable_filter

xfs 939662 3

libcrc32c 12644 1 xfs

sd_mod 45497 6

crc_t10dif 12714 2 target_core_mod,sd_mod

crct10dif_generic 12647 0

crct10dif_pclmul 14289 1

crct10dif_common 12595 3 crct10dif_pclmul,crct10dif_generic,crc_t10dif

crc32c_intel 22079 1

ahci 29907 3

libahci 32031 1 ahci

igb 192580 0

ptp 19231 1 igb

pps_core 19106 1 ptp

i2c_algo_bit 13413 1 igb

i2c_core 40582 2 igb,i2c_algo_bit

dca 15130 1 igb

libata 218730 2 ahci,libahci

uio 19259 1 igb_uio

The messages are:

[root@SNIPER app]# ./testpmd -c 0xffffffffff -n 4 – -i --rxq=32 --txq=32 --nb-cores=32 --port-topology=paired

EAL: Detected 56 lcore(s)

EAL: Detected 2 NUMA nodes

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket

EAL: No available hugepages reported in hugepages-1048576kB

EAL: Probing VFIO support…

EAL: PCI device 0000:03:00.0 on NUMA socket 0

EAL: probe driver: 8086:1533 net_e1000_igb

EAL: PCI device 0000:04:00.0 on NUMA socket 0

EAL: probe driver: 8086:1533 net_e1000_igb

EAL: PCI device 0000:3b:00.0 on NUMA socket 0

EAL: probe driver: 15b3:1017 net_mlx5

net_mlx5: no Verbs device matches PCI device 0000:3b:00.0, are kernel drivers loaded?

EAL: Requested device 0000:3b:00.0 cannot be used

EAL: PCI device 0000:60:00.0 on NUMA socket 0

EAL: probe driver: 8086:37d3 net_i40e

EAL: PCI device 0000:60:00.1 on NUMA socket 0

EAL: probe driver: 8086:37d3 net_i40e

EAL: PCI device 0000:60:00.2 on NUMA socket 0

EAL: probe driver: 8086:37d3 net_i40e

EAL: PCI device 0000:60:00.3 on NUMA socket 0

EAL: probe driver: 8086:37d3 net_i40e

EAL: PCI device 0000:af:00.0 on NUMA socket 1

EAL: probe driver: 15b3:1017 net_mlx5

net_mlx5: no Verbs device matches PCI device 0000:af:00.0, are kernel drivers loaded?

EAL: Requested device 0000:af:00.0 cannot be used

testpmd: No probed ethernet devices

Interactive-mode selected

testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=459456, size=2176, socket=0

testpmd: preferred mempool ops selected: ring_mp_mc

testpmd: create a new mbuf pool <mbuf_pool_socket_1>: n=459456, size=2176, socket=1

testpmd: preferred mempool ops selected: ring_mp_mc

Done

Hi ,

There are 2 ways to use DPDK with the MLX5_PMD

  1. RDMA Core library with a recent enough Linux kernel release (recommended)

  2. Or Mellanox OFED/EN, which provides compatibility with older releases.

You can refer to section “32.6.1.1. RDMA Core with Linux Kernel” in the MLX5_PMD driver guide

http://doc.dpdk.org/guides/nics/mlx5.html?highlight=mlx5

Thanks,

Samer

Hi,

  1. RDMA Core library with a recent enough Linux kernel release (recommended)

I’d like to use 1) method in 3.10.0 kernel. In guide manual, I should upgrade the kernel, but my system can’t be upgraded.

Is it possible to use 1) method in 3.10.0 kernel?