How can I rebuild the OFED driver on Ubuntu with configure option --with-pa-mr?

I need the physical memory option to work with the IB library and I’m having problems building a custom version of the OFED as suggested by the physical address memory region document. The build document linked is for building RPM packages and its not relevant for a Debian system. I can extract the sources from the OFED .iso, configure the option and build it, but then its a mystery on how to install it. I’ve tried creating my own .iso and using the ./mlnxofedinstall, but I can’t get the actual binaries compiled with the pa-mr to install. I’m checking using /etc/infiniband/info and all the standard options show up, but no --with-pa-mr. It seems like the install scripts pre-configure the options, but its not clear to me where that is done and how it can be changed. If anybody has any info, I’d greatly appreciate it.

Hello Scott,

I does seem the instructions are only for RPMs. Could you try building RPMs via the instructions and then converting the RPMs to DEBs via: https://www.rosehosting.com/blog/how-to-install-rpm-packages-on-ubuntu/

Many thanks,

~Steve

I can build and convert the RPM packages, but then what? They don’t match the .deb’s that are in the OFED source .tgz and I can’t install them. They fail to install, presumably because there are conflicts with the existing packages installed. Its only a partial OFED package, so I can’t uninstall, then install them and then install the rest of the OFED. This is really frustrating and I’m sure theres a way to use the tools to create Debian install packages, but it just doesn’t appear to be documented anywhere.

Hello Scott,

cd /opt/MLNX_OFED_LINUX-4.6-1.0.1.1-ubuntu19.04-x86_64/src

tar xvzf MLNX_OFED_SRC-4.6-1.0.1.1.tgz

cd /MLNX_OFED_SRC-4.6-1.0.1.1/SOURCES/mlnx-ofed-kernel-4.6

./configure --with-pa-mr

make

Please let me know you results…

thanks - Steve

Correction,

When Mellanox OFED installed, kernel source located in /usr/src/mlnx-ofa-kernel-X.Y directory. Reconfigure it and run make:

etc/infiniband/info | awk -F ‘:’ ‘/Configure/ {print $2 " --with-pa-mr"}’

make

make install

You may want to backup existing /lib/modules/uname -r directory before the change.

Thank-you guys. I had built this directory before, but did not think to then try ‘make install’. There are multiple src directories and I wasn’t sure if this was all that was needed for this to work.

I was able to get this to work via the last set of instructions. Interesting note: the info script, /etc/infiniband/info, does not properly report the option ‘–with-pa-mr’. The code does now work though so I can tell it had an actual effect. Thanks again!