How to enable the driver for the ConnectX-3 PRO NIC?

The server have some Connect-X 3 pro NICs. Recently, I have to do a Benchmark for RoCE.

So I download the driver MLNX_OFED-5.0-1.0.0.0 and extract, then , I run the install scripts as follow :

sudo ./mlnxofedinstall --basic

after that,

sudo ./etc/init.d/openibd restart

but , the NIC don’t worked.

my os is Ubuntu18.04, and the SPID is IBM1200111023, and the fireware is 2.34.5000?.

what’s the reason? I need your help. thanks.

I resolve it by myself. This tutorial has been verified on Linux

  1. Why can’t Connect-x 3 pro from IBM be enabled for RoCE after installing the MLNX_OFED driver?

The reason is that the firmware is an IBM custom firmware, which only supports the TCP protocol of the Ethernet protocol, and no IB transport layer protocol of RoCE.

You can use mstflint or mlxburn to query the infomation of rom. In addition, the firmware of this network card is 2.345.000, and the latest firmware version required by the mlx driver is 2.42.5000.

  1. Solution: burn the latest firmware. If you use mlxup, the official tool of mlx, you will encounter a PSID incompatibility problem at this time, and you cannot burn it into.

a) Download and install MLNX_OFED, which is mainly used to install mlxfint or mlxburn

b) Download the 2.42.5000 firmware provided by mlx : fw-2.4.5000.bin

c) Run the lspci command to find the address of the mlx network card in the form : 00: 01.0

d) Run the command and record the returned guid and mac

sudo mlxburn -dev 00: 01.0 q

e) Run the command:

sudo mlxburn -dev 00: 01.0 -image fw-2.4.5000.bin -use_image_rom -allow_psid_change -dev_type 4103

or the following command

sudo mstflint -d 00: 01.0 -i fw-2.4.5000.bin --use_image_rom --allow_psid_change burn

f)Run the command:

sudo mlxburn -dev 00: 01.0 q,

you can see that there are two versions of the firmware, the old firmware shows that it is running

g) Run the command

sudo /etc/init.d/openibd restart

h) Run the command

sudo reboot