Rping maximum size

I’m trying to verify RDMA connectivity between two machines using rping. It works with message size up to 1444 bytes, but fails above that. Where does this number come from? I’m trying to get ksmbd to work, and some folks there suggested I need to make sure rping works with 32K message size. It looks like the server received the ping properly, but the client does not receive a response.

Each machine has a ConnectX-6. The linux server uses MLNX-EN driver 5.7 installed with --vma option. The Windows client uses WinOF2 driver 2.90.

Server output:

# rping -s -vVd -S 32768
size 32768
created cm_id 0x56213786f490
rdma_bind_addr successful
rdma_listen
cma_event type RDMA_CM_EVENT_CONNECT_REQUEST cma_id 0x7f9ef0000ce0 (child)
child cma 0x7f9ef0000ce0
created pd 0x56213786f820
created channel 0x562137864ef0
created cq 0x56213786f880
created qp 0x56213786fae8
rping_setup_buffers called on cb 0x562137863830
allocated & registered buffers...
accepting client connection request
cq_thread started.
cma_event type RDMA_CM_EVENT_ESTABLISHED cma_id 0x7f9ef0000ce0 (child)
ESTABLISHED
recv completion
Received rkey 4f34 addr 1a15a4c3240 len 1445 from peer
server received sink adv
server posted rdma read req 
rdma read completion
server received read complete
server ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefg
server posted go ahead
send completion
recv completion
Received rkey 4b29 addr 1a15a4cfa10 len 1445 from peer
server received sink adv
rdma write from lkey 4706 laddr 56213787e0a0 len 1445
cq completion failed status 12
wait for RDMA_WRITE_COMPLETE state 11
rping server failed: -1
rping_free_buffers called on cb 0x562137863830
destroy cm_id 0x56213786f490
root@lanos-moonshine:/home/netflix#

Client output:

> nd_rping -c -vVd -C 1 -S 1445 -a 10.49.54.21
wait for RDMA_WRITE_COMPLETE state 8
rping client failed: -1073741823