mlnx driver mdev.ko module conflicts with NVIDIA grid drivers (CentOS)

Infrastructure & Networking Software And Drivers
1

Hi,

the mellanox driver comes with its own mdev.ko kernel module, which overrides the default one (the one that the OS came with):

/lib/modules/3.10.0-1062.el7.x86_64/kernel/drivers/vfio/mdev/mdev.ko.xz

becomes

/lib/modules/3.10.0-1062.el7.x86_64/extra/mlnx-ofa_kernel/drivers/vfio/mdev/mdev.ko

But NVIDIA grid driver also uses mdev.ko to create vGPUs, and is unable to do so with mlnx-ofa_kernel version of mdev.ko

On the other hand, using mellanox driver with the default mdev.ko.xz results in some weird behavior in SR-IOV mode. For example, on some VFs (passed through to VMs) in IPoIB environment packets arrive “scrambled”.

What is the proper way to deal with this issue?

Any help would be appreciated! Thanks!

2

kernel module and mellanox kernel model locate on different folder. If you don’t use mellanox kernel module just remove it.

/usr/lib/modules/3.10.0-693.el7.x86_64/kernel/drivers/vfio/mdev/mdev.ko.xz

/usr/lib/modules/3.10.0-862.14.4.el7.x86_64/extra/mlnx-ofa_kernel/drivers/vfio/mdev/mdev.ko

And when you install OFED you can filter the package by below,

Extra package filtering:

–kernel-only Install kernel space packages only

–user-space-only Filter selected packages and install only User Space packages

–without- Do not install package

–with- Force installing package