They are set to Ethernet in the drivers tools. Their ethernet ports show up fine in Windows Server (We are on 2019).
Regardless, both ports on both card fail to detect all our passive DAC cables. Said cables work fine on a MCX314A-BCCT.
The cables are from FS.com, with a transceiver encoded for Mellanox NIC.
The chances that both ports on both card would fail is quite unlikely. What options are we missing that makes those cards report no cable attached?
For some reason, the passive DAC isn’t properly detected when in 10Gb Ethernet mode, and the MCX354A-QCBTB is only supporting 40Gb InfiniBand or 10Gb Ethernet.
We flashed to the MCX354A-FCB firmware (That supports 40Gb Ethernet), and now everything works fine.
We contacted the maker of the transceivers to see why they don’t work in 10Gb mode.
While now everything should be in 40Gb mode, our iPerf test shows it’s probably not really running 40Gb.
I was under the impression both MCX354A-QCBTB and MCX354A-FCB were the same hardware with different firmware. Is the MCX354A-QCBTB hardware limited to 10Gb regardless if it’s handshake to 40?