ConnectX Pro EN 313-BCCT how to get 40Gbe

iperf gives me max of 18Gb/s with th nic’s direct attached

and even worse 8Gb/s with the sx1012 interconnect

hw: z800 LnkSta 5GT/s x 8 asus z10pe-d8 intel xeon server LnkSta 8GT/s x 8

os: 2 x centos6.7

can someone help an ethernet basic skilled user


If you card sits on a GEN2 slot you will not be able to reach 40Gb/s, you should make sure it sits on a Gen3 slot on both servers

“lspci | awk ‘/Mellanox/ {print $1}’ | sudo xargs -i -r lspci -s {} -xxxvvv”

in general for troubleshooting steps you should first start with the below after confirming your hardware is set to support the right speeds :

Share how much you’re getting with multi-thread test (-P 12), what driver, firmware versions you have. and if there any drops reported by our nic on the Receiver ?

Driver and Firmware version : ethtool -i ethX

ethtool stats, preferable after the OS came after boot. capture from the receiver : ethtool -S ethX | egrep ‘drop|err|sto|paus’