I have Microsoft Windows 2012 R2 cluster and some nodes have ConnectX-3 adapters and some nodes have ConnectX-4 Lx adapters.
There is RoCE connectivity between nodes with ConnectX-4 Lx adapters, but there isn’t connectivity between nodes with different adapters.
I think it’s because ConnectX-3 adapters use RoCE 1.0 mode, but ConnectX-4 Lx adapters use RoCE 2.0 mode.
I tred to change RoCE mode from 1.0 to 2.0 for ConnectX-3 adapters by “Set-MlnxDriverCoreSetting -RoceMode 2”, but had warning – “SingleFunc_2_0_0: RoCE v2.0 mode was requested, but it is not supported. The NIC starts in RoCE v1.5 mode” and RoCE connectivity doesn’t work.
What the best way to fix my problem ?
ConnectX-3 adapters don’t work in RoCE 2.0 mode at all ? How about new FW ? Today I have WinOF-5_35 with FW:
Image type: FS2
FW Version: 2.40.5032
FW Release Date: 16.1.2017
Product Version: 02.40.50.32
Rom Info: type=PXE version=3.4.747 devid=4099
Device ID: 4099
Description: Node Port1 Port2 Sys image
GUIDs: ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff
MACs: e41d2ddfa540 e41d2ddfa541
I don’t find way to change to RoCE 1.0 mode for ConnectX-4 Lx adapters for Microsoft Windows environment.