Strange HCAs found online

Infrastructure & Networking Adapters And Cables
#1

Hello!

Recently I bought a pair of Mellanox HCAs for my homelab.

When I plug it in in my home server running ESXi 6.7 I can see

Mellanox Technologies MT27710 Family [ConnectX-4 Lx Flash Recovery]

Seller claimed that this pair is MCX4121A-ACAT Mellanox ConnectX-4 25GbE

I tried to burn latest FW on this card from CentOS. After that I can see in ESXi with esxcfg-nics -l but nothing in GUI

I installed this card to my WIndows machine and queried it with flint -d q

I shows N/A in GUIDs and MACs

I tried to modify FW file with GUID and flash it again. Now I can see MAC address when I press CTRL-B during boot and with ipconfig /all, but flint query still doesn’t show GUID and MAC

I removed heatsink and checked part number on card IC. It says mt27711a0-fdcf-ge

According to this table http://www.mellanox.com/related-docs/prod_silicon/PB_ConnectX-4_Lx-EN_IC.pdf it should be Single-port 50GbE adapter (25*2=50?), but is does look like 25GE and it has two SFP28 ports.

So, I put card back to ESXi and now I can see it in GUI, link is established (10G, my switch is Quanta LB6M flashed to Broadcom TurboIron 24X).

I passed through 1 port of HCA to my CentOS VM. Seems that it works, it got address from my DHCP server and I was able to connect by SSH to VM.

Here is output from some commands:

mst status

MST modules:

MST PCI module is not loaded

MST PCI configuration module loaded

MST devices:

/dev/mst/mt4117_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:0b:00.0 addr.reg=88 data.reg=92

Chip revision is: 00

flint -d /dev/mst/mt4117_pciconf0 q

Image type: FS3

FW Version: 14.24.1000

FW Release Date: 26.11.2018

Product Version: 14.24.1000

Rom Info: type=UEFI version=14.17.11 cpu=AMD64,AARCH64

type=PXE version=3.5.603 cpu=AMD64

Description: UID GuidsNumber

Base GUID: N/A 4

Base MAC: N/A 4

Image VSD: N/A

Device VSD: N/A

PSID: MT_2420110034

Security Attributes: N/A

mlxfwmanager

Querying Mellanox devices firmware …

Device #1:

Device Type: ConnectX4LX

Part Number: MCX4121A-ACA_Ax

Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6

PSID: MT_2420110034

PCI Device Name: /dev/mst/mt4117_pciconf0

Base MAC: N/A

Versions: Current Available

FW 14.24.1000 14.24.1000

PXE 3.5.0603 3.5.0603

UEFI 14.17.0011 14.17.0011

Status: Up to date

So, my question is - did I do everything right way and this adapter can be used or I missed something?

I have another similar adapter I would like to restore. Any suggestions or advice?

#2

Hi Dmitry,

The adapter can work as expected but without MAC/GUID.

For basic functionality you can run the following commands:

For Linux:

  1. lspci | grep Mell

  2. ifconfig

  3. ibstat

  4. ethtool commands

For ESXi:

esxcli software vib list | grep ^net-mlx

esxcli system module list | grep ^mlx

esxcli system module parameters list -m mlx5_core (****in case of CX-4/Lx)

esxcli network nic list

Mellanox NICs statistics:

esxcli network nic stats get -n vmnicX

ethtool –i vmnicX (****all information on driver ver., fw, device-number)

Note: we always recommend to use the latest driver/firmware/MFT versions

Firmware:

http://www.mellanox.com/page/management_tools

http://www.mellanox.com/page/firmware_download

Driver:

http://www.mellanox.com/page/software_overview_ib

http://www.mellanox.com/page/software_overview_eth

In case you would like to burn the Original Mac address/GUID

please open a case by sending an email to support@mellanox.com

and we will be happy to assist.

Thanks,

Samer

#3

Dear Samer!

Thanks you for quick reply! I will immediately send an email to Mellanox Support!

#4

So, support just replied to me that they don’t have active support contract and I have to use community portal (

The only action they advised is

flint -d -guid e41d2d0300570fc0 -mac 0000e41d2d570fc0 -ocr sg

And it didn’t work. MAC changed, but flint still shows N/A in MAC and GUID

So, anybody please help me with these problems