NFSoRDMA: svcrdma: Error -12 posting RDMA_READ


I configured the NFSoRDMA according to this document. HowTo Configure NFS over RDMA (RoCE)

This is the same problem, but it does not help me. ConnectX-4 LX: NFSoRDMA failures in MOFED-3.3

Here’s my question:

After the configuration, start fio testing,The read test will succeed and the write test will fail !

INFO: NFS SERVER|imageView2/2/w/1240

INFO: NFS CLIENT:|imageView2/2/w/1240

Any suggestions on it ?




I tried to upgrade the kernel to 3.16.46 ( MLNX_OFED_LINUX-3.4- ) , but it did not work.

I tried to upgrade the driver to MLNX_OFED_LINUX-4.1- (kernel: 3.16.46), but The kernel module (xprtrdma) does not seem to match the driver version.

alkx Can you help me and give me some tips ? Thank you

Hi guoqingwang,

Thank you for reaching out to the Mellanox Support Community and for your patience in this matter.

Unfortunately, as from Mellanox OFED version 3.4-x and above NFSoRDMA is not supported anymore. This was never mentioned in the Release Notes for version 3.4- but it is mentioned in later released version.

If you want to use NFSoRDMA, you need to downgrade the driver to Mellanox OFED version 3.3- in which we do still support NFSoRDMA and is available for Ubuntu 14.04 or switch to the OS-vendor supplied driver (INBOX Driver) for the Mellanox NIC.

Hopefully this will clarify and resolves your issue.




Hi Martijn :

thank you very much for your reply

After downgraded the driver to Mellanox OFED version 3.3- from your advice, I have recompiled the mlnx-ofed-kernel module with nfsrdma support .

But still get same error “svcrdma: Error -12 posting RDMA_READ” while executing “fio --rw=randwrite“.

I wonder if there is a bug or some wrong configuration .

Hope to get any of your suggestions .





HowTo Compile MLNX_OFED Drivers (mlnx-ofa_kernel example)

options taken from /etc/infiniband/info + –with-nfsrdma-mod

mlnx-ofed-kernel-3.3# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod --with-mlx4_en-mod --with-mlx5-mod --with-ipoib-mod --with-sdp-mod --with-e_ipoib-mod –with-nfsrdma-mod

mlnx-ofed-kernel-3.3# make

copy xprtrdma.ko to nfs client and svcrdma.ko to nfs server

mlnx-ofed-kernel-3.3# cp net/sunrpc/xprtrdma/xprtrdma.ko /lib/modules/3.16.46-031646-generic/updates/dkms/

mlnx-ofed-kernel-3.3# depmod -a && modprobe xprtrdma