Help Identifying Card

Hello! I recently acquired some new-in-box OEM Dell R640 servers that came with Mellanox ConnectX-5 cards installed. I’m hoping someone may be able to identify these and tell me if they are OEM or retail, and what firmware they should use. According the the service tag, I’m not seeing these as installed by Dell, but then again as an OEM server I suppose it could have been custom built. Card info below.

Model No: NV303212A
ConnectX-5 25GbE w/FPGA
P/N MNV303212A-ADLT
S/N MT1935X08342

There is also another label with the following: DP/N 0NMD3R

Windows Device Info: PCI\VEN_15B3&DEV_1017&SUBSYS_004615B3&REV_00

On 2 of them, I attempted to install the latest drivers and update firmware for Windows 2022 following the instructions from the Nvidia support section: Windows Driver Installation - ConnectX-5 Ethernet - NVIDIA Networking Docs (Windows Server 2022).

However, after performing the update and rebooting the server, the card is disabled with the following message: A PCIe link training failure is observed in Slot1 and device link is disabled mellanox R640

I have follow Dell’s instructions to try and clear the error, but I’m afraid the firmware update may be the culprit. If I swap with the non-updated ones, I can still boot per normal, and the device info shows the following firmware version: 16.25.1020

Is there a way I can reflash the updated ones?

Also, my only real use for a 25GbE card would be for direct connection between servers for synchronization and iSCSI channels (Starwind VSAN). If these cards aren’t intended or suited for that, please let me know if possible.

Thanks in advance for any info and help!

Hello,

What drivers and guidelines have you followed?

Did you install our latest WinOF2 driver version 2.80?

If you flashed the FW, I am assuming our MFT package has been installed.

(Mellanox Firmware Tools (MFT))

What is the PSID of the HCA card in order to identify if OEM or not?

What FW did you upgraded from 16.25.1020?

Did you upgrade to a FW that is supported according to our WinOF2 release note for driver version 2.80?

mst status -v

flint -d q

After upgrading and regardless the PCIe event generated, is the HCA card still visible under mst status -v?

If yes, you can downgrade the FW the same way with our flint utility pointing to a different binary file.

(> flint -d -I fw.bin b followed by a reboot or mlxfwreset)

What is the protocol on the HCA card?

Where does it plug into?

Is the card plugged into a PCI4.0/3.0 slot?

Is it reproducible on a different server running Windows 2022?

Is it reproducible on a supported Linux server? Same FW upgrade?

Are you using supported cable(s) based on the FW release note?

Sophie.

Hello! Thank you for your reply. I’ll try and answer what I can.

The cards are installed in PCIe 3.0 slots on a Dell R440 server running Windows Server 2022.
They had the following firmware installed to begin with: 16.25.1020
I followed this guide: Windows Driver Installation - ConnectX-5 Ethernet - NVIDIA Networking Docs
I selected the option to ‘Upgrade the HCA’s firmware version (Recommended)’ as described in the guide above.
I used WinOF-2 v2.80 for Windows Server 2022: https://www.mellanox.com/downloads/WinOF/MLNX_WinOF2-2_80_50000_All_x64.exe
After reboot, I get a message from server saying: UEFI0067: A PCIe link training failure is observed in Slot1 and device link is disabled.
Because the device link is disabled, I am unable to see it once Windows boots.

I have an identical card that was not updated that still is visible in Windows.
PCI\VEN_15B3&DEV_1017&SUBSYS_004615B3&REV_00\6&15B57994&0&00800000

These cards aren’t in use yet; but I’m trying to determine vendor/support in case I’m able to.
I have installed the MFT exe for Windows but have not used it yet.
Please let me know if I can provide anything else useful. Thank you!

According to the documentation, I do not see Windows showing as a supported OS for this card.

System Requirements
Operating
Systems/
Distributions
RHEL
Ubuntu
SLES
For version information, please refer to the Mellanox Innova-2 Flex Open Bundle
Release Notes.
Please refer to Vivado release notes for supported Operating Systems.

https://docs.nvidia.com/networking/display/Innova2Flex/Recommended+Documents+and+Tools

Thank you. So this is not a standard ConnectX-5 card then?

Not it is not😊

Thank you; I think this solved my question of whether or not I have use for them. I also found this: NVIDIA INNOVA-2 FLEX OPEN ADAPTER CARD - NVIDIA INNOVA-2 FLEX OPEN ADAPTER CARD - NVIDIA Networking Docs.

As for the 2 cards I updated - are those salvageable? I’ll likely want to sell these if not being used.