How to get Mellanox ConnectX-6 Dx cards to work in Supermicro server

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Hi,

We have a Supermicro SSG-121E-NE316R 1U server running Ubuntu 22.04 LTS server with 2 Mellanox MT28908 family ConnectX-6 NICs.
I’ve installed the MLNX_OFED_LINUX 5.8-4.1.5.0-LTS driver. The OS sees the Mellanox cards and when I use ethtool to check each interface, it shows “Link detected: yes.” The Mellanox interfaces also show as state UP.
This leads me to believe all is well, and I assign static IPs to the Mellanox interfaces, making sure to also add the network gateway in the default route, and while I can ping the IP address that is assigned to the interface I can’t ping the gateway, or reach any other node on the network.
We’ve popped in another basic network card, and the OS immediately recognizes the card, and I can assign a static IP, and immediately it’s able to ping the gateway and go out onto our network.

What are we missing for the Mellanox ConnectX-6 DX cards to work on their own without a backup NIC?

2

Sounds like a driver issue. I wonder if you can ping 127.0.0.1 successfully.

3

hi,
Generally, it should work after you config like that.
Suggest to check:

  1. if there are some errors in dmesg , especially if there is an error log related to mlx5
  2. check the route setting(ex, ARP)

Good luck,
Levei