Interconnection between HDR and FDR switches

Hi there. We just bought a new cluster with 8790 HDR switch recently, and I wanted to integrate the old cluster as well. Since the nodes of the old clusters are connected to a 6025 FDR switch, so I just connected the two switches with two EDR cables. Everything else was fine, but iblinkinfo suggests that the connection between two switches are just 10Gb

24 35 ==( 4X 2.5 Gbps Active/ LinkUp)==> 11 79 “Quantum Mellanox Technologies” ( Could be 14.0625 Gbps)

24 36 ==( 4X 2.5 Gbps Active/ LinkUp)==> 11 77 “Quantum Mellanox Technologies” ( Could be 14.0625 Gbps)

ibdiagnet also suggests following error:

Speed / Width checks

-I- Link Speed Check (Compare to supported link speed)

-E- Links Speed Check finished with errors

-E- Link: Sec0d9a03002c89b0/Nec0d9a03002c89b0/P35<–>Sb8599f0300c66f52/Nb8599f0300c66f52/P40 (79) - Unexpected actual link speed 2.5 (enable_speed1=“2.5 or 5 or 10 or 14 or FDR10”, enable_speed2=“2.5 or 10 or 25 or 50” therefore final speed should be 10)

-E- Link: Sec0d9a03002c89b0/Nec0d9a03002c89b0/P36<–>Sb8599f0300c66f52/Nb8599f0300c66f52/P39 (77) - Unexpected actual link speed 2.5 (enable_speed1=“2.5 or 5 or 10 or 14 or FDR10”, enable_speed2=“2.5 or 10 or 25 or 50” therefore final speed should be 10)

Did I miss anything? Thanks in advance.

Hello Chao,

Thank you for posting your inquiry on the NVIDIA Networking Community.

Based on the information provided, please make sure that the latest f/w and s/w code is installed on the HDR and FDR components.

Also when connecting FDR to HDR components, please review the supported connectivity matrix → https://docs.mellanox.com/display/MellanoxQuantumFirmwarev2720081604/Firmware+Compatible+Products

As you mentioned you use EDR cables, unfortunately this is unsupported. Supported cables for FDR to HDR switch connectivity are FDR DAC cables only.

Thank you and regards,

~NVIDIA Networking Technical Support

After updating the firmware to the latest version and using FDR cable, I confirm that now the link between two switches are of 56 Gb/s. Thanks a lot. :)

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.