Does CX3-pro supports guest RDMA in hyper-V linux VMs?

[1989chenguo]

Dear friends

I have bought several CX3-pro cards. I’m now not able to use RDMA in my guest Ubuntu VM.

I’m wondering whether this card supports guest RDMA or I’m making some mistakes in configuration?

My detailed configuration is as follows:

Host Configuration: Windows server 2019 on Dell R720XD, WinOF driver 5.50.42, NIC firmware 2.42.5000. RDMA and sriov both enabled in BIOS, NIC, and HyperV swith and VM.

Guest Configuration: Ubuntu 16.04 in Hyper-V. Driver “MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu16.04-x86_64”.

In my guest Ubuntu VM, I can see the Mellanox NIC VF through lspci

and ibv_devinfo reports fine

but mst status reports there is no device found

and ib_send_bw also fails

[aviap]

if you approach to our Mellanox WinOF Release-Note v5.50, https://docs.mellanox.com/display/WINOFv55052000/Release+Notes+Rev+5.50.52000

There you can find that within Windows 2019 Host- “Any supported guest OS for Hyper-V”, including Ubuntu 16.04 , is supported only in Virtualization Mode of VMQ, meaning: it is on the top on vmSwitch with no RDMA capabilities!!

The only guests (VMs) that WinOF v5.50 supports with RDMA capabilities are Windows-2016/2019 OS

In other words - I’m afarid you cannot use guest RDMA in your Ubuntu 16.04 VM, for the resouns I mentioned above

Here is a good article that will guide you through how to configure guest-RDMA on Windows 2019 host, which also involves configuring Hyper-V SET (switch embedded team)

https://www.dell.com/support/article/us/en/04/how16693/how-to-configure-guest-rdma-on-windows-server-2019?lang=en