krping (4.7 kernel version) crashing with mlx5_core (CX415A), it is passing with mlx4_core(CX354A)

Hi,

I am trying to run krping code which I got from the below link on the Linux kernel 4.7

GitHub - larrystevenwise/krping: Kernel Mode RDMA Ping https://github.com/larrystevenwise/krping

with 4.7 kernel I can able to use mlx4 card (). But it is failing with mlx5 100G (cx415A). It is giving crash from mlx5_core. Please find below logs. Please let me know if you need any further information.

KRPING Client Log:

[ 701.240641] krping_init

[ 713.858099] krping: proc write

|client,addr=192.168.69.127,port=9999,count=100|

[ 713.858106] client

[ 713.858109] ipaddr (192.168.69.127)

[ 713.858112] port 9999

[ 713.858114] count 100

[ 713.858122] created cm_id ffff880242e9b000

[ 713.858170] cma_event type 0 cma_id ffff880242e9b000

(parent)

[ 713.858271] cma_event type 2 cma_id ffff880242e9b000

(parent)

[ 713.858282] rdma_resolve_addr - rdma_resolve_route

successful

[ 713.858464] created pd ffff880095581cc0

[ 713.858812] created cq ffff880252c53a00

[ 713.859223] created qp ffff8802530a3800

[ 713.859225] krping: krping_setup_buffers called on

cb ffff880242e99c00

[ 713.859316] krping: allocated & registered

buffers…

[ 713.861548] cma_event type 9 cma_id ffff880242e9b000

(parent)

[ 713.861555] ESTABLISHED

[ 713.861569] rdma_connect successful

[ 713.861575] RDMA addr 24614b640 rkey 1563 len 64

[ 713.861963] mlx5_0:dump_cqe:263:(pid 0): dump error

cqe

[ 713.861967] 00000000 00000000 00000000 00000000

[ 713.861968] 00000000 00000000 00000000 00000000

[ 713.861970] 00000000 00000000 00000000 00000000

[ 713.861972] 00000000 92005204 0a0000b6 000055d2

[ 713.861975] krping: cq completion failed with wr_id

0 status 4 opcode -1 vender_err 52

[ 713.861987] krping: wait for RDMA_WRITE_ADV state 10

[ 713.862082] krping_free_buffers called on cb

ffff880242e99c00

[ 713.863131] destroy cm_id ffff880242e9b000

KR PING SERVER LOG:

[ 816.542447] krping_init

[ 846.465708] krping: proc write

|server,addr=192.168.69.127,port=9999|

[ 846.465717] server

[ 846.465721] ipaddr (192.168.69.127)

[ 846.465725] port 9999

[ 846.465738] created cm_id ffff8804112ffc00

[ 846.465750] rdma_bind_addr successful

[ 846.465752] rdma_listen

[ 852.111109] cma_event type 4 cma_id ffff880429de4800

(child)

[ 852.111116] child cma ffff880429de4800

[ 852.111379] created pd ffff8804112ed440

[ 852.111594] created cq ffff88041470c800

[ 852.112100] created qp ffff880429205800

[ 852.112106] krping: krping_setup_buffers called on

cb ffff8804112fe800

[ 852.112216] krping: allocated & registered

buffers…

[ 852.112218] accepting client connection request

[ 852.113286] cma_event type 9 cma_id ffff880429de4800

(child)

[ 852.113290] ESTABLISHED

[ 852.113813] cma_event type 10 cma_id

ffff880429de4800 (child)

[ 852.113818] krping: DISCONNECT EVENT…

[ 852.113894] krping: wait for CONNECTED state 10

[ 852.113899] krping: connect error -1

[ 852.113903] krping_free_buffers called on cb

ffff8804112fe800

[ 852.115259] destroy cm_id ffff8804112ffc00

[root@xhdipsnvme1 krping-master]# cat /proc/krping

can you add dmesg to the question.

i might have a solution for this.

krping is not part of Mellanox OFED. Could you contact ‘kpring’ developers and check if they can point on any issue with mlx5 driver?

Hi Yonatan,

The logs what ever are there, they are dmesg outputs