I’m using a SN2100 switch and everything seems to work fine with ConnectX-5 cards. I was just trying to add another older server using ConnectX-3 cards, which I expected to work at 56Gbe, but I get no link. I’ve installed all of the latest drivers and firmware, and I’m using MCP1600 cables. The cards are 354A-FCBT. Both ports are configured for etrhernet:
/sbin/connectx_port_config -s
Port configuration for PCI device: 0000:06:00.0 is:
eth
eth# ethtool p4p1
Settings for p4p1:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
40000baseCR4/Full
40000baseSR4/Full
56000baseCR4/Full
56000baseSR4/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
40000baseCR4/Full
40000baseSR4/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000014 (20)
link ifdown
Link detected: no
ethtool p4p2
Settings for p4p2:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
40000baseCR4/Full
40000baseSR4/Full
56000baseCR4/Full
56000baseSR4/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
40000baseCR4/Full
40000baseSR4/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000014 (20)
link ifdown
Link detected: no
hca_self_test.ofed
---- Performing Adapter Device Self Test ----
Number of CAs Detected … 1
PCI Device Check … PASS
Kernel Arch … x86_64
Host Driver Version … MLNX_OFED_LINUX-4.6-1.0.1.1 (OFED-4.6-1.0.1): 3.10.0-957.el7.x86_64
Host Driver RPM Check … PASS
Firmware on CA #0 VPI … v2.42.5000
Host Driver Initialization … PASS
Number of CA Ports Active … 0
Error Counter Check on CA #0 (VPI)… PASS
Kernel Syslog Check … PASS
Node GUID on CA #0 (VPI) … NA
------------------ DONE ---------------------
Is this configuration not supported? Any idea how to get a link up if it is?