@Martijn van Breugel I have now upgraded to fw-ConnectX4-rel-12_25_1020-MCX456A-ECA_Ax-UEFI-14.18.19-FlexBoot-3.5.701 which I assume is the fw that you were referring to, but there is no change:
Card to self - 100G (with C & E cable)
Card to other Card (CX5) - 100G (with C & E cable)
Card to Switch - no connect unless switch set to 40G
CX5 to switch -100G (with C & E cable)
Do you have any other ideas?
If this is not the card but the switch (which seems a possibility given the above results) - would you think this is a hardware or a software issue?
Hardware would be unsolvable for me, software I could try to work on. Otoh, it happens with both OcNos and SoNIC, so also hard to understand…
Sorry for the late reply. I was on vacation the last 2 weeks.
It seems that it becomes an interoperability issue between the ConnectX-4 VPI and the switch.
I think the best thing to do is to open a support case directly with Mellanox Technical Support, so we can continue to debug on a deeper level.
As you are based in EMEA, our EMEA colleagues will assist you but I will be following the case.
Hi @Martijn van Breugel ,
somehow notifications are not working as expected:/
I took the liberty to reopen the original case - can you please re-assign appropriately?
Many thanks for your help,
I finally managed to resolve this issue this week.
It turns out that CX4’s (and Chelsio’s) don’t seem to be able to properly auto-negotiate FEC settings on Sonic (and OcNos) switches [or maybe its the Broadcom or Celestica HW].
When manually configuring FEC to setting RS (instead of none or FC) the CX4’s linked up with all cables types I tried (Mellanox C and E types and also Dell branded ones).