NFS over RDMA on Centos 7.3


i am trying to install NFS over RDMA on a centos 7.3 with Mellanox.

I installed MLNX_OFED_LINUX-4.0- and configured a nfs server.

Kernel: 3.10.0-514.16.1.el7.x86_64

If i connect to the server via ip, everything is fine. If i try to add rdma support in nfs, i retrieve following errors while starting:

rpc.nfsd[3423]: rpc.nfsd: Unable to request RDMA services: Protocol not supported

My lsmod shows:

lsmod | grep rdma

rpcrdma 12430 0

svcrdma 12430 0

rdma_ucm 22741 0

rdma_cm 54793 1 rdma_ucm

iw_cm 42241 1 rdma_cm

ib_cm 51558 3 rdma_cm,ib_ucm,ib_ipoib

ib_uverbs 75283 2 ib_ucm,rdma_ucm

ib_core 240474 10 rdma_cm,ib_cm,iw_cm,mlx4_ib,mlx5_ib,ib_ucm,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib

mlx_compat 16849 16 rdma_cm,ib_cm,iw_cm,rpcrdma,svcrdma,mlx4_en,mlx4_ib,mlx5_ib,ib_ucm,ib_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib

Also i tried to add port 20049 manualy to /proc/fs/nfsd/portlist

echo “rdma 20049” > /proc/fs/nfsd/portlist

-bash: echo: write error: Protocol not supported

RDMA tests looks good:

front01:/etc/infiniband # rdma_server

rdma_server: start

rdma_server: end 0

Speedtests also give results up to 50Gbit.

Any ideas how i can setup NFS over RDMA or what i am missing?



NFSoRDMA is not supported in the Mellanox OFED version 3.4 and higher. However, inbox OS driver can be used if NFSoRDMA support is necessary.

Why was NFSoRDMA support dropped?