Mellanox Connectx-6 Dx adapter slow only 6gbps Throughput

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

I have 2 servers specifically Lenovo SR635 both with Mellanox Connectx-6 Dx OCP 100G network cards.
One can transfer data speed at high throughputs and one is stuck at 6.5gbps. It wont go any higher than 6.5gbps.
The cpus and memory and os configurations are the same.
I can’t figure out why its stuck at such a speed.

2

Could you check the PCI bandwidth? What is the system on Lenovo SR635?
If it’s a Linux system, we can use ‘lspci -vvv -s ’ to check the lnkCap / lnkSta
How do you test it, back-to-back or is there a switch in between?
Is there a limitation on the switch side?
Maybe you can swap the low and high throughput network interface cards in the two servers to isolate the issue first.