Using MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64, the DKMS rpcrdma.ko kernel module is not being created

Infrastructure & Networking Software And Drivers Mellanox OFED
1

I previously installed MLNX_OFED_LINUX-24.01-0.3.3.1-ubuntu20.04-x86_64 on an Ubuntu 20.04 system using mlnxofedinstall. This compiled a DKMS version of rpcrdma.ko as shown below:

root@kerkyra:/lib/modules# find . -name rpcrdma\*
./5.15.0-100-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko
./5.15.0-100-generic/updates/dkms/rpcrdma.ko

However, installing MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64 on a nearly identical Ubuntu 20.04 system, also using mlnxofedinstall, did not result in the same outcome:

root@zakynthos:/lib/modules/5.15.0-113-generic# find . -name rpcrdma\*
./kernel/net/sunrpc/xprtrdma/rpcrdma.ko

The problem, it turns out is that the ~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64/DEBS/mlnx-nfsrdma-dkms_24.04.OFED.24.04.0.6.6.1-1_all.deb package is not being automatically installed any more and needs to be installed manually:

dpkg -i MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64/DEBS/mlnx-nfsrdma-dkms_24.04.OFED.24.04.0.6.6.1-1_all.deb

Not sure if this is intentional, but reporting it just in case this is an oversight, as this behavior diverges from the previous MLNX_OFED_LINUX-24 collection.

2

Hello @pgoetz1,

Thank you for posting your query on our community. I ran a quick test on my setup and I see similar behavior with both driver versions (DKMS compiled version is not there) as you can see below:

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# cat /etc/os-release
NAME=“Ubuntu”
VERSION=“20.04.5 LTS (Focal Fossa)”
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=“Ubuntu 20.04.5 LTS”
VERSION_ID=“20.04”
HOME_URL=“https://www.ubuntu.com/”
SUPPORT_URL=“https://help.ubuntu.com/”
BUG_REPORT_URL=“Bugs : Ubuntu”
PRIVACY_POLICY_URL=“Data privacy | Ubuntu”
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# ofed_info -s
MLNX_OFED_LINUX-24.04-0.6.6.0:

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# cd /usr/lib/modules
root@l-csi-0113l:/usr/lib/modules# find . -name rpcrdma*
./5.4.0-181-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko
./5.4.0-187-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko

NAME=“Ubuntu”
VERSION=“20.04.5 LTS (Focal Fossa)”
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=“Ubuntu 20.04.5 LTS”
VERSION_ID=“20.04”
HOME_URL=“https://www.ubuntu.com/”
SUPPORT_URL=“https://help.ubuntu.com/”
BUG_REPORT_URL=“Bugs : Ubuntu”
PRIVACY_POLICY_URL=“Data privacy | Ubuntu”
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@l-csi-0113l:~# ofed_info -s
MLNX_OFED_LINUX-24.01-0.3.3.1:

root@l-csi-0113l:/usr/lib/modules# find . -name rpcrdma*
./5.4.0-181-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko
./5.4.0-187-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko

In order to install the relevant modules for NFS over RDMA, an extra flag is required for the mlnxofedinstall command: --with-nfsrdma. Once you install the drivers with this flag, you should be able to see the DKMS compiled version as below:

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# ./mlnxofedinstall --with-nfsrdma
Logs dir: /tmp/MLNX_OFED_LINUX.1155688.logs
General log file: /tmp/MLNX_OFED_LINUX.1155688.logs/general.log

Below is the list of MLNX_OFED_LINUX packages that you have chosen
(some may have been added by the installer due to package dependencies):

ofed-scripts
mlnx-tools
mlnx-ofed-kernel-utils
mlnx-ofed-kernel-dkms
iser-dkms

…
…

Installation passed successfully
To load the new driver, run:
/etc/init.d/openibd restart

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# ofed_info -s
MLNX_OFED_LINUX-24.04-0.6.6.0:

root@l-csi-0113l:~/MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu20.04-x86_64# cd /usr/lib/modules
root@l-csi-0113l:/usr/lib/modules# find . -name rpcrdma*
./5.4.0-181-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko
./5.4.0-187-generic/kernel/net/sunrpc/xprtrdma/rpcrdma.ko
./5.4.0-187-generic/updates/dkms/rpcrdma.ko

Thanks,
Bhargavi