Unable to Use 5W QSFP28 Module with ConnectX-5(MCX555A-ECA_Ax_Bx)

Hi, I’m experiencing an issue with my Mellanox ConnectX-5 NIC (MCX555A-ECA_Ax_Bx) and a QSFP28 module (INPHI CORP IN-Q2AY2-20-M1). The NIC is not enabling the High Power Class (>3.5W) for the module, which is capable of 5.0W operation.

Current status:

  • NIC Firmware version: 16.35.3502
  • QSFP28 module is recognized but operates in low power mode
  • ethtool -m shows: “5.0W max. Power consumption, High Power Class (> 3.5 W) not enabled”
  • PCIe link is operating at full capacity (Speed 8GT/s, Width x16)
  • DISABLE_SLOT_POWER_LIMITER and ADVANCED_POWER_SETTINGS are both set to True(1)

I’ve tried:

  • Adjusting NIC settings using mlxconfig
  • Resetting the NIC using mlxfwreset
  • Checking BIOS settings for PCIe power

However, I haven’t found a way to enable the high power mode for the QSFP module. Is there a specific setting or procedure to enable the 5.0W power class for this NIC and QSFP module combination? Any assistance or guidance would be greatly appreciated."

$ sudo ethtool -m enp23s0np0
	Identifier                                : 0x11 (QSFP28)
	Extended identifier                       : 0xcf
	Extended identifier description           : 3.5W max. Power consumption
	Extended identifier description           : CDR present in TX, CDR present in RX
	Extended identifier description           : 5.0W max. Power consumption,  High Power Class (> 3.5 W) not enabled
	Power set                                 : On
	Power override                            : On
	Connector                                 : 0x07 (LC)
	Transceiver codes                         : 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00
	Transceiver type                          : 100GE-DWDM2 (DWDM transceiver using 2 wavelengths on a 1550 nm DWDM grid with a reach up to 80 km)
	Encoding                                  : 0x08 (PAM4)
	BR, Nominal                               : 25500Mbps
	Rate identifier                           : 0x00
	Length (SMF,km)                           : 80km
	Length (OM3 50um)                         : 0m
	Length (OM2 50um)                         : 0m
	Length (OM1 62.5um)                       : 0m
	Length (Copper or Active cable)           : 0m
	Transmitter technology                    : 0x50 (1550 nm DFB)
	Laser wavelength                          : 1561.400nm
	Laser wavelength tolerance                : 0.025nm
	Vendor name                               : INPHI CORP
	Vendor OUI                                : 00:21:b8
	Vendor PN                                 : IN-Q2AY2-20-M1
	Vendor rev                                : 10
	Vendor SN                                 : L2XXXXXXXX
	Date code                                 : 210330
	Revision Compliance                       : SFF-8636 Rev 2.5/2.6/2.7
	Module temperature                        : 0.00 degrees C / 32.00 degrees F
	Module voltage                            : 3.3507 V
$ sudo mlxconfig -d /dev/mst/mt4119_pciconf0 q

Device #1:
----------

Device type:        ConnectX5           
Name:               MCX555A-ECA_Ax_Bx   
Description:        ConnectX-5 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device:             /dev/mst/mt4119_pciconf0
        DISABLE_SLOT_POWER_LIMITER                  True(1)             
        ADVANCED_POWER_SETTINGS                     True(1)             
        AUTO_POWER_SAVE_LINK_DOWN_P1                False(0)   

Hello hattorishingo3379,

Welcome, and thank you for posting your inquiry to the NVIDIA Developer Forums!

Unfortunately, third-party cables/transceivers are not supported with our adapters, unless they are specifically noted in the ConnectX-5 firmware release notes (link). Different manufacturers have different specifications, our adapters are not tested with all of them.

If you have a valid support contract, and are experiencing these issues with supported cables/transceivers (per that link above), we highly recommend opening a ticket with our NVIDIA Enterprise Experience team for further triage.

Best,
NVIDIA Enterprise Experience

1 Like