Unable to Force Link Speed for ConnectX-4 Lx on Windows

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

I’m troubleshooting a ConnectX-4 Lx stuck at 1Gbps on a machine that was previously running fine at 40Gbps with ConnectX-3.

I can’t seem to manually set the link speed:

>Mlx5Cmd.exe -LinkSpeed -Name "Ethernet 2" -Query
Link is down
Couldn't query the linkspeeds supported by cable since the link is down

LinkSpeeds supported by device only:
1 Gbps
10 Gbps
25 Gbps
40 Gbps


>Mlx5Cmd.exe -LinkSpeed -Name "Ethernet 2" -Set 1
ERROR: set command is not supported.

>Mlx5Cmd.exe -LinkSpeed -Name "Ethernet 2" -Set 10
ERROR: set command is not supported.

>Mlx5Cmd.exe -LinkSpeed -Name "Ethernet 2" -Set 25
ERROR: set command is not supported.

>Mlx5Cmd.exe -LinkSpeed -Name "Ethernet 2" -Set 40
ERROR: set command is not supported.

How do I get this to work? Thanks!

Adapter info:

Driver Version : 3.10.25798.0
Firmware Version : 14.32.1010
Port Number : 1
Bus Type : PCI-E 8.0 GT/s x4
Link Speed : ----
Part Number : MCX4131A-BCAT
Device Id : 4117
Revision Id : 0
Network Status : Disconnected
Adapter Friendly Name : Ethernet 2
Port Type : ETH

2

Why are you trying to force a link speed?

Our HCA(s) by default is set to auto-negotiate (It should be a plug and play)

What is the switch that HCA/port connects to?

Is the setting on that switch port to AN or no AN (Aka force speed)?

Are you using a supported cable for the CX4 LX based on the supported speed of that HCA?

Is the main goal to bring the link up or determine why mlx5cmd.exe LinkSpeed/Set not working?

I don’t have a Windows lab setup handy though I would suggest opening a support case should it needs to be investigated further.

Also, comparing CX3 & CX4 LX is not a good comparison, two different beast…

3

I previously had two ConnectX-3 MCX354A’s using Kaiam XQX2502’s directly connected together at 40Gbps. There is no switch involved.

When I upgraded the machine to Alder Lake, the ConnectX-3 Windows drivers would BSOD when the E-cores were active.

Swapping a ConnectX-4 MCX4131A-BCAT in fixed the BSOD, but failed to connect at 40Gbps with the same equipment.

I wanted to set the card explicitly to 40Gbps to see if that would fix the issue. Is there a known issue with setting the link speed?

4

I’m having the exact same issues. It’d be nice if there was a real reason why the “set command is not supsported”.

5

The solution is to run mlx5cmd in an elevated shell. If you run it without elevation, it will output the above opaque error.

6

I’m having a similar issue and came across this thread. CX4 Lx won’t link with a 10gbe sfp+ port on a switch. Trying to manually set the link speed on the card to see if that will resolve the issue.

Using the set command in mlx5cmd also gives me the “set command is not supported” error even in an elevated shell. Looking through the winof2 documentation I don’t see what the issue could be.

7

I have the same problem - did you ever find a solution?