I’ve just installed a MCX4121A-ACAT ConnectX-4 Lx EN NIC in a Linux PC (x86_64, Linux Mint 19.3 (based on Ubuntu 18.04), kernel 5.4.0-72).
The NIC is connected to an external device which supports only 10Gbase-R (e.g. 10Gbase-SR, 10Gbase-CR etc. but not 10Gbase-KR). A link appears to be established, but no data appears to be transmitted or received.
‘ethtool’ reports the following when using a copper cable / transceiver in the 1st SFP28 cage on the MCX4121A-ACAT :
Settings for enp1s0f0:
Supported ports: [ Backplane ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
25000baseCR/Full
25000baseKR/Full
25000baseSR/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: None BaseR
Advertised link modes: 10000baseKR/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Advertised FEC modes: None
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
Similarly, when using an optical cable / transceiver in the 2nd SFP28 cage :
Settings for enp1s0f1:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
25000baseCR/Full
25000baseKR/Full
25000baseSR/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: None BaseR
Advertised link modes: 10000baseKR/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Advertised FEC modes: None
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
Thus, it appears that the MCX4121A-ACAT supports base-SR, base-CR and base-KR at 25G, but only base-KR at 10G. Is this correct ?
Further information which may be useful :
- the MCX4121A-ACAT firmware version is 14.27.1016. Is there any possibility that upgrading to the newest firmware (14.30.1004) is likely to provide support for any other 10G link modes ?
- output from lspci -v :
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
Subsystem: Mellanox Technologies Stand-up ConnectX-4 Lx EN, 25GbE dual-port SFP28, PCIe3.0 x8, MCX4121A-ACAT
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at a6000000 (64-bit, prefetchable) [size=32M]
Expansion ROM at a5400000 [disabled] [size=1M]
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [48] Vital Product Data
Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
Capabilities: [c0] Vendor Specific Information: Len=18 <?>
Capabilities: [40] Power Management version 3
Capabilities: [100] Advanced Error Reporting
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
Capabilities: [1c0] #19
Capabilities: [230] Access Control Services
Kernel driver in use: mlx5_core
Kernel modules: mlx5_core
01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
Subsystem: Mellanox Technologies Stand-up ConnectX-4 Lx EN, 25GbE dual-port SFP28, PCIe3.0 x8, MCX4121A-ACAT
Flags: bus master, fast devsel, latency 0, IRQ 17
Memory at a8000000 (64-bit, prefetchable) [size=32M]
Expansion ROM at a5300000 [disabled] [size=1M]
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [48] Vital Product Data
Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
Capabilities: [c0] Vendor Specific Information: Len=18 <?>
Capabilities: [40] Power Management version 3
Capabilities: [100] Advanced Error Reporting
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
Capabilities: [230] Access Control Services
Kernel driver in use: mlx5_core
Kernel modules: mlx5_core
Thank you,
Andrew.