The server have some Connect-X 3 pro NICs. Recently, I have to do a Benchmark for RoCE.
So I download the driver MLNX_OFED-5.0-1.0.0.0 and extract, then , I run the install scripts as follow :
sudo ./mlnxofedinstall --basic
after that,
sudo ./etc/init.d/openibd restart
but , the NIC don’t worked.
my os is Ubuntu18.04, and the SPID is IBM1200111023, and the fireware is 2.34.5000?.
what’s the reason? I need your help. thanks.
I resolve it by myself. This tutorial has been verified on Linux
- Why can’t Connect-x 3 pro from IBM be enabled for RoCE after installing the MLNX_OFED driver?
The reason is that the firmware is an IBM custom firmware, which only supports the TCP protocol of the Ethernet protocol, and no IB transport layer protocol of RoCE.
You can use mstflint or mlxburn to query the infomation of rom. In addition, the firmware of this network card is 2.345.000, and the latest firmware version required by the mlx driver is 2.42.5000.
- Solution: burn the latest firmware. If you use mlxup, the official tool of mlx, you will encounter a PSID incompatibility problem at this time, and you cannot burn it into.
a) Download and install MLNX_OFED, which is mainly used to install mlxfint or mlxburn
b) Download the 2.42.5000 firmware provided by mlx : fw-2.4.5000.bin
c) Run the lspci command to find the address of the mlx network card in the form : 00: 01.0
d) Run the command and record the returned guid and mac
sudo mlxburn -dev 00: 01.0 q
e) Run the command:
sudo mlxburn -dev 00: 01.0 -image fw-2.4.5000.bin -use_image_rom -allow_psid_change -dev_type 4103
or the following command
sudo mstflint -d 00: 01.0 -i fw-2.4.5000.bin --use_image_rom --allow_psid_change burn
f)Run the command:
sudo mlxburn -dev 00: 01.0 q,
you can see that there are two versions of the firmware, the old firmware shows that it is running
g) Run the command
sudo /etc/init.d/openibd restart
h) Run the command
sudo reboot