I manage a small compute cluster that uses Mellanox ConnectX 3 InfiniBand Cards and runs Ubuntu.
Mellanox OFED driver support for ConnectX 3/ 3 Pro cards is now limited to the LTS drivers. While NFSoRDMA is supported in the Mellanox OFED LTS driver for Ubuntu, it is limited to the 18.04 LTS version.
My question is can support for NFSoRDMA be added to the Mellanox OFED LTS driver for the most recent LTS release of Ubuntu vice the 2 year old version. I would like to update our cluster for security reasons but support for NFSoRDMA and the OFED driver is the limitation preventing this. We would use the inbox driver but performance benchmarks show the Mellanox driver is superior for our HPC workloads.
The 5.1x OFED drivers support NFSoRDMA on Ubuntu 20.04. Had the choice to move support for ConnectX 3 hardware to the LTS driver come two releases later (some number of months), your ConnectX 3 customers would have support for Ubuntu 20.04.
Ubuntu is the only nonEnterprise distribution for which NFSoRDMA support is provided. It seems reasonable for Mellanox to support major performance features, such as NFSoRDMA, for major releases of at least one OS distribution on Mellanox hardware until the EOS milestone is reached. This would allow for your customers to not have to compromise between performance and cyber security while using your hardware within its support timeline. In this case we are only talking a single additional version as the next LTS release, Ubuntu 22.04, would seem to be beyond the EOS milestone for ConnectX 3 hardware.
I do not understand Mellanox reasoning here and wish you would reconsider. I will soon be forced to upgrade to 20.04 for security reasons (IT policy outside my control) and my users will lose performance compared to what they already have today. This is not for corner-case use either, performance of file transfers across the cluster affects many aspects of everyday use of the cluster for my users.
I understand. The next OFED LTS version should be released in a few months. I will forward your request to the relevant team to check if support for Ubuntu 20.4 can be added to this upcoming release.
Regardless the list of supported OS versions that is mentioned in the release-notes, have you tried to use NFSoRDMA on Ubuntu 20.04 and saw any issue?
I think that the NFSoRDMA module would likely work fine for the most part. However, I cannot test this as the mlnxofedinstall script responds with the following when the installer is run with the --with-nfsrdma option, “WARNING: NFSoRDMA is not supported over kernel 5.4.0-51-generic, will continue installation without it”.
I do not know how to force the installer to install the NFSoRDMA module.