How to properly disable ConnectX-7 auto-negotiation and force port settings?

How to properly DISABLE ConnectX-7 auto-negotiation and force port settings?
I need to disable specifically for my configuration… (whole other story)

set mst_dev=“/dev/mst/mt4129_pciconf1”

Using RHEL8.8 hardware configuration contains ConnectX7 MCX755106AC-HEA_Ax with MMA1B00-C100D

I currently mlxconfig -d $mst_dev query
[…]
Device type: ConnectX7
Name: MCX755106AC-HEA_Ax
Description: NVIDIA ConnectX-7 HHHL adapter Card; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe 5.0 x16 with x16 PCIe extension option; Crypto Enabled; Secure Boot Enabled;
[…]
mlxlink -d $mst_dev --show_module --show_device --show_fec -c -e
[…]
Tool Information

Firmware Version : 28.40.1000
amBER Version : 3.2
MFT Version : mft 4.28.0-92
[…]
Module Info

Identifier : QSFP28
Compliance : 100G AOC or 25GAUI C2M AOC. No FEC
Cable Technology : 850 nm VCSEL
Cable Type : Active cable (active copper / optics)
OUI : Mellanox
Vendor Name : Mellanox
Vendor Part Number : MMA1B00-C100D
[…]

I would like to disable auto-negotiation and force the following port settings for 100G_4X
Speed: 100G
Width: 4x
FEC: Standard RS-FEC - RS(528,514)

Here’s the commands that I run with sudo (root) permission:
mlxconfig -d $mst_dev -y set KEEP_ETH_LINK_UP_P1=FALSE
mlxlink -d $mst_dev --port_state DN
mlxlink -d $mst_dev --loopback NO
mlxconfig -d $mst_dev -y set PHY_AUTO_NEG_P1=AUTO_NEG_DISABLED
mlxlink -d $mst_dev --speeds 100G_4X
mlxconfig -d $mst_dev -y set PHY_RATE_MASK_OVERRIDE_P1=TRUE
# 0x200 for Bit 9 - CAUI-4 / 100GBASE-CR4 / KR4
mlxconfig -d $mst_dev -y set PHY_RATE_MASK_P1=0x200
mlxconfig -d $mst_dev -y set LINK_TYPE_P1=ETH
mlxconfig -d $mst_dev -y set KEEP_ETH_LINK_UP_P1=FALSE
mlxconfig -d $mst_dev -y set AUTO_POWER_SAVE_LINK_DOWN_P1=FALSE
mlxlink -d $mst_dev --speeds 100G_4X
mlxlink -d $mst_dev --link_mode_force --speeds 100G_4X
mlxlink -d $mst_dev --fec RS --fec_speed 100G
# 0x2: MODE_2 Force use of Standard RS-FEC - RS(528,514) <<PICK THIS
# 0x4: MODE_4 - disable non_dme (enable Auto Negotiation)
mlxconfig -d $mst_dev -y set PHY_FEC_OVERRIDE_P1=0x2

-l (Reset levels) Run reset with the specified reset-level.

0: Driver, PCI link, network link will remain up (“live-Patch”)

3: Driver restart and PCI reset

4: Warm Reboot

-t Reset-types (relevant only for reset-levels 3, 4):

0: Full chip reset

1: Phy-less reset (“port-alive” - network link will remain up)

2: NIC only reset (for SoC devices)

mlxfwreset -d $mst_dev -y -l 3 -t 0 r 
mlxlink -d $mst_dev --port_state UP

important this must be done while the port_state is up (For some reason does not stick if done earlier than here!)

Note. the 100G is overall speed while _4x is the number of lanes used to support the speed.

mlxlink -d $mst_dev --link_mode_force --speeds 100G_4X

With the above configuration settings, I am initially getting the following error reported by mlxlink:
mlxlink -d /dev/mst/mt4129_pciconf1 --show_module --show_device --show_fec -c -e

Operational Info

State : e[33mPollinge[0m
Physical state : e[33mETH_AN_FSM_ENABLEe[0m
Speed : e[33mN/Ae[0m
Width : e[33mN/Ae[0m
FEC : e[33mN/Ae[0m
Loopback Mode : e[32mNo Loopbacke[0m
Auto Negotiation : e[34mFORCE - 100G_4Xe[0m

Supported Info

Enabled Link Speed (Ext.) : e[32m0x00000200 (100G_4X)e[0m
Supported Cable Speed (Ext.) : e[32m0x000002f2 (100G_4X,50G_2X,40G,25G,10G,1G)e[0m

Troubleshooting Info

Status Opcode : e[31m2e[0m
Group Opcode : e[31mPHY FWe[0m
Recommendation : e[31mNegotiation failuree[0m
[…]

within a few seconds I get the following status:
mlxlink -d /dev/mst/mt4129_pciconf1 --show_module --show_device --show_fec -c -e

Operational Info

State : e[32mActivee[0m
Physical state : e[32mETH_AN_FSM_ENABLEe[0m
Speed : e[32m100Ge[0m
Width : e[32m4xe[0m
FEC : e[32mStandard RS-FEC - RS(528,514)e[0m
Loopback Mode : e[32mNo Loopbacke[0m
Auto Negotiation : e[34mFORCE - 100G_4Xe[0m

Supported Info

Enabled Link Speed (Ext.) : e[32m0x00000200 (100G_4X)e[0m
Supported Cable Speed (Ext.) : e[32m0x000002f2 (100G_4X,50G_2X,40G,25G,10G,1G)e[0m

Troubleshooting Info

Status Opcode : e[31m15e[0m
Group Opcode : e[31mPHY FWe[0m
Recommendation : e[31mBad signal integritye[0m

Tool Information

Firmware Version : e[32m28.40.1000e[0m
amBER Version : e[32m3.2e[0m
MFT Version : e[32mmft 4.28.0-92e[0m

Module Info

Identifier : QSFP28
Compliance : 100G AOC or 25GAUI C2M AOC. No FEC
[…]


My questions are:

  1. Why is the module_info displayed “Compliance: 100G AOC or 25GAUI” C2M AOC. No FEC" (for EXT_ETHERNET_COMPLIANCE_CODE_100G_AOC=0x18) instead of the expected transceiver MMA1B00-C100D EEPROM value “Compliance: 100GBASE-SR4 or 25GBASE-SR” (for EXT_ETHERNET_COMPLIANCE_CODE_100GBASE_SR4=0x2) ?

  2. Does the following indicate the adapter is still attempting to auto-negotiate?
    Troubleshooting Info


Status Opcode : 2
Group Opcode : PHY FW
Recommendation : Negotiation failure

  1. What commands or modifications are required to properly force the link to disable AUTO-negotiation?

Attached are full log files related to my question.
mt4129_pciconf1_amber_2025-01-14_16-48-37_.csv.txt (7.4 KB)
mt4129_pciconf1_mlxlink_5953_query_2025-01-14_16-48-37_.txt (46.1 KB)
eth4_mlxlink_5953_ethtoool_2025-01-14_16-48-37_.txt (10.3 KB)
mt4129_pciconf1_mlxcables_5953_query_2025-01-14_16-48-37_.txt (16.2 KB)