ConnectX-8 NIC split to 8 total 10G ports

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

I would like to use breakout cables to connect multiple devices with 10G ports to a ConnectX-8 C8240 network card with qsfp to sfp breakout cables. I see that this NIC supports configuration for 8 Ethernet ports. I have run the mlxconfig commands but only see 5 ports (vs 2 originally) and have not been able to get the ports to negotiate. Help would be appreciated as I’m very new to this.

sudo mlxlink -d /dev/mst/mt4131_pciconf0.3

Operational Info
----------------
State                              : Receiver detect
Physical state                     : ETH_AN_FSM_ENABLE
Speed                              : N/A
Width                              : N/A
FEC                                : N/A
Loopback Mode                      : No Loopback 
Auto Negotiation                   : ON 

Supported Info
--------------
Enabled Link Speed (Ext.)          : 0x00000952 (100G_1X,50G_1X,25G,10G,1G) 
Supported Cable Speed (Ext.)       : 0x00000032 (40G,10G,1G) 

Troubleshooting Info
--------------------
Status Opcode                      : 2 
Group Opcode                       : PHY FW 
Recommendation                     : Negotiation failure 

Tool Information
----------------
Firmware Version                   : 40.45.1202 
amBER Version                      : 6.4 
MFT Version                        : 4.35.0-159
3

Hi

  1. Each QSFP112 port on C8240 is built from 4 × 100 G PAM4 electrical lanes
  2. These lanes are designed to:

    • Aggregate upward (400 G / 2×400 G)

    • Split only to supported Ethernet breakouts (e.g., 2×200 G or 4×100 G)

Thanks,

Suo

4

Are you sure? This page seems to indicate that they can support NRZ and lower rates: Linking DGXs, Switches, ConnectX-7 & ConnectX-8 Adapters, and BlueField-3 DPUs Overview - NVIDIA Docs

“4-channel QSFP112 (4x100G-PAM4) cages can support QSFP56 (4x50G-PAM4) and QSFP28 (4x25G-NRZ) devices”

I’m not sure this means what I think it means; I’m happy to be corrected. Thanks for the reply and any future guidance.

5

Hi William,

Yes, it can support 4x25G NRZ.

However, you can only split CX8 standard card port into four ports due to a hardware limitation.

Thanks,

Suo