Link Down between ConnectX6-Dx and Cisco Nexus 5624Q

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
, ,
1

Hello

I am currently having difficulty activating the link between the ConnectX6 Dx and the Cisco Switch (Nexus 5624Q).

At first I thought it was an auto-negotiation issue, so I manually set the link speed and duplex of the NIC and switch.

CISCO Switch

switch(config) # interface ethernet 1/1
switch(config) # speed 40000
switch(config) # duplex full

ConnectX6 Dx

sudo ethtool enp2s0f0np0 speed 40000 autoneg off

However, the link is still down and the details are shown below.

$ sudo mlxlink -d /dev/mst/mt4125_pciconf0

Operational Info
----------------
State                           : Polling
Physical state                  : ETH_AN_FSM_ENABLE
Speed                           : N/A
Width                           : N/A
FEC                             : N/A
Loopback Mode                   : No Loopback
Auto Negotiation                : FORCE - 40G

Supported Info
--------------
Enabled Link Speed (Ext.)       : 0x00000020 (40G)
Supported Cable Speed (Ext.)    : 0x00000032 (40G,10G,1G)

Troubleshooting Info
--------------------
Status Opcode                   : 36
Group Opcode                    : PHY FW
Recommendation                  : Other issues

Tool Information
----------------
Firmware Version                : 22.35.2000
amBER Version                   : 2.08
MFT Version                     : mft 4.22.1-307

I would like to know what type of error the PHY FW group code is. Additionally, if you have any experience with connecting ConnectX6 NICs to Cisco switches, I would appreciate your help.

Any comments would be very helpful.

Best Regards
Taekyoung

2

Hi ,

Please make sure you are using supported module transceiver
https://docs.nvidia.com/networking/display/connectx6dxfirmwarev22391002/firmware+compatible+products#src-144705761_FirmwareCompatibleProducts-Supported3rdPartyCablesandModules

Please provide "sudo mlxlink -d /dev/mst/mt4125_pciconf0 -m -e -c "

Thanks
Samer

3

Hi Samer

We resolved this issue by disabling auto-negotiation for both devices.

We really appreciate your willingness to help.

Best Regards
Taekyoung