Changing parameters for Soft RoCE

Hi,

I’m using Soft RoCE running on regular Intel 1G on-board NIC, and trying to connect to NVMe over RoCEv2 target. Linux software base NVMe/RoCEv2 target works fine so my setup should be OK but other native NVMe/RoCEv2 target doesn’t work.

nvme discover -t rdma -a 192.168.1.4 -s 4420

Failed to write to /dev/nvme-fabrics: Connection timed out

NVMe/RoCEv2 target device supplier is saying, it could be because Soft RoCE specifies 0x80 in Responder Resources in ConnectRequest packet. It can only supports up to 64 (0x40). Is there a way to change parameter, Responder Resources in SoftRoCE.

I tried to set parameter from nvme-cli but it doesn’t change anything. And target supplier is saying it should be parameter in RoCE.

nvme discover -t rdma -a -s 4420 -Q 64

nvme discover -t rdma -a -s 4420 -i 64

I wonder it could be one of parameters under

/sys/module/rdma_rxe/parameters/

based on the following website. But somehow I was not able to set/change parameter for it.

It doesn’t allow me to edit file.

http://manpages.ubuntu.com/manpages/bionic/man7/rxe.7.html

Let me know if there’s any suggestion here…

I attached screenshot of wireshark packet.

Thanks in advance,

Shiro

Hi Shiro,

Based on your description, it does not appear to have any Mellanox components related (HCA/Driver),

I would suggest to post your question via Ubuntu/nvme community forum.

Sophie.