Problem with nfs over rdma and ubuntu 22.04

Infrastructure & Networking Switches and Gateways InfiniBand/VPI Switch Systems
1

I have the same problem described here: MLNX OFED driver build failure for ubuntu-22.04 with latest kernel 5.15.0.84-generic

It seems that there is a problem in the mlnx-nfsrdma-dkms_5.8-OFED.5.8.3.0.4.1_all.deb package that makes it impossible to install in ubuntu 22.04.

The equivalent in earlier versions of ubuntu installs as expected, but not this version (downloaded from the 22.04 iso)

The installation generates the following error:

DKMS make.log for mlnx-nfsrdma-5.8 for kernel 5.15.0-88-generic (x86_64)
Thu Nov 16 14:22:15 UTC 2023
/bin/sh: 1: Syntax error: Unterminated quoted string
/bin/sh: 1: [: -lt: unexpected operator
grep: mlnx-nfsrdma.spec: No such file or directory
make -C /lib/modules/5.15.0-88-generic/build O=/lib/modules/5.15.0-88-generic/build M=/var/lib/dkms/mlnx-nfsrdma/5.8/build KBUILD_EXTRA_SYMBOLS=/usr/src/ofa_kernel/x86_64/5.15.0-88-generic/Module.symvers \
        CONFIG_SUNRPC_XPRT_RDMA=m \
        CONFIG_SUNRPC_XPRT_RDMA_CLIENT=m \
        CONFIG_SUNRPC_XPRT_RDMA_SERVER=m \
        CONFIG_SUNRPC_XPRT_RDMA_DUMMY= \
        CONFIG_GPU_DIRECT_STORAGE=y \
        CONFIG_DTRACE= \
        CONFIG_CTF= \
        LINUXINCLUDE=' -include /lib/modules/5.15.0-88-generic/build/include/generated/autoconf.h -include /lib/modules/5.15.0-88-generic/build/include/linux/kconfig.h -include /usr/src/ofa_kernel/x86_64/5.15.0-8
...
/var/lib/dkms/mlnx-nfsrdma/5.8/build/svc_rdma_transport.c:110:10: error: 'const struct svc_xprt_ops' has no member named 'xpo_release_rqst'; did you mean 'xpo_release_ctxt'?
  110 |         .xpo_release_rqst = svc_rdma_release_rqst,
      |          ^~~~~~~~~~~~~~~~
      |          xpo_release_ctxt
/var/lib/dkms/mlnx-nfsrdma/5.8/build/svc_rdma_transport.c:110:29: error: initialization of 'void (*)(struct svc_xprt *, void *)' from incompatible pointer type 'void (*)(struct svc_rqst *)' [-Werror=incompatible-pointer-types]
  110 |         .xpo_release_rqst = svc_rdma_release_rqst,
      |                             ^~~~~~~~~~~~~~~~~~~~~
/var/lib/dkms/mlnx-nfsrdma/5.8/build/svc_rdma_transport.c:110:29: note: (near initialization for 'svc_rdma_ops.xpo_release_ctxt')
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:297: /var/lib/dkms/mlnx-nfsrdma/5.8/build/svc_rdma_transport.o] Error 1
make[1]: *** [Makefile:1909: /var/lib/dkms/mlnx-nfsrdma/5.8/build] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.15.0-88-generic'
make: *** [makefile:105: default] Error 2

I have tried to install it using mlnxofedinstall --with-nfsrdma, but also installing the package with dpkg -i, and the results are the same.

Any help will be appreciated.

2

I have unpacked the .deb and tried to compile it by myself. The result is the same:

  CC [M]  /tmp/ttt/usr/src/mlnx-nfsrdma-5.8/svc_rdma_transport.o
/tmp/ttt/usr/src/mlnx-nfsrdma-5.8/svc_rdma_transport.c:110:10: error: ‘const struct svc_xprt_ops’ has no member named ‘xpo_release_rqst’; did you mean ‘xpo_release_ctxt’?
  110 |         .xpo_release_rqst = svc_rdma_release_rqst,
      |          ^~~~~~~~~~~~~~~~
      |          xpo_release_ctxt
/tmp/ttt/usr/src/mlnx-nfsrdma-5.8/svc_rdma_transport.c:110:29: error: initialization of ‘void (*)(struct svc_xprt *, void *)’ from incompatible pointer type ‘void (*)(struct svc_rqst *)’ [-Werror=incompatible-pointer-types]
  110 |         .xpo_release_rqst = svc_rdma_release_rqst,
      |                             ^~~~~~~~~~~~~~~~~~~~~
/tmp/ttt/usr/src/mlnx-nfsrdma-5.8/svc_rdma_transport.c:110:29: note: (near initialization for ‘svc_rdma_ops.xpo_release_ctxt’)
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:297: /tmp/ttt/usr/src/mlnx-nfsrdma-5.8/svc_rdma_transport.o] Error 1
make[1]: *** [Makefile:1909: /tmp/ttt/usr/src/mlnx-nfsrdma-5.8] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.15.0-88-generic'
make: *** [makefile:89: default] Error 2
3

I have been digging a bit and it seems that the problem has to do with the kernel version. It seems that the up-to-date kernel version 5.15.0-88 does not work, but version 5.19.0-50 works.

So my solution was to simply upgraded to that kernel (it seems that they reverted the change from xpo_release_rqst to xpo_release_ctxt) in file linux/sunrpc/svc_rdma.h

In previous versions of the kernel (e.g. 5.15.0-47) the package also worked.

1 Like
4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.