How to get Mellanox ConnectX-6 Dx cards to work in Supermicro server


We have a Supermicro SSG-121E-NE316R 1U server running Ubuntu 22.04 LTS server with 2 Mellanox MT28908 family ConnectX-6 NICs.
I’ve installed the MLNX_OFED_LINUX 5.8- driver. The OS sees the Mellanox cards and when I use ethtool to check each interface, it shows “Link detected: yes.” The Mellanox interfaces also show as state UP.
This leads me to believe all is well, and I assign static IPs to the Mellanox interfaces, making sure to also add the network gateway in the default route, and while I can ping the IP address that is assigned to the interface I can’t ping the gateway, or reach any other node on the network.
We’ve popped in another basic network card, and the OS immediately recognizes the card, and I can assign a static IP, and immediately it’s able to ping the gateway and go out onto our network.

What are we missing for the Mellanox ConnectX-6 DX cards to work on their own without a backup NIC?

Sounds like a driver issue. I wonder if you can ping successfully.