ConnectX-2 IPoIB Performance Help

I really need some expertise here:

I have two Windows 10 Machines with two MHQH19B-XTR 40 Gbit Adapters and a QSFP cable in between. The Vlan manager is opensm.

The connection should be about 32Gbits Lan. In reality i only get 5 Gbit performance. So clearly something is very wrong.

C:\Program Files\Mellanox\MLNX_VPI\IB\Tools>iblinkinfo

CA: E8400:

0x0002c903004cdfb1 2 1 ==( 4X 10.0 Gbps Active/ LinkUp)==> 1 1 “IP35” ( )

CA: IP35:

0x0002c903004ef325 1 1 ==( 4X 10.0 Gbps Active/ LinkUp)==> 2 1 “E8400” ( )

I tested my IPoIB with a program called lanbench and nd_read_bw:

nd_read_bw -a -n 100 -C 169.254.195.189

#qp #bytes #iterations MR [Mmps] Gb/s CPU Util.

0 512 100 0.843 3.45 0.00

0 1024 100 0.629 5.15 0.00

0 2048 100 0.313 5.13 0.00

0 4096 100 0.165 5.39 0.00

0 8192 100 0.083 5.44 0.00

0 16384 100 0.042 5.47 0.00

0 32768 100 0.021 5.47 100.00

…stays at 5.47 after that. with CPU util 100%

The processor is an intel core I7 4790k so it should not be at 100%. According to Taskmanager only 1 Core is actively used.

Firmware, Drivers, Windows 10 are up to date.

My goal is to get the fastest possible File sharing between two windows 10 machines.

What could be the problem here and how do I fix it?

Hi Daniel,

  1. Confirm current Mellanox OFED Driver installed supports the Window version, Connectx-2 HCA card with a valid FW. (RN of Winof).

  2. Download the Performance Tuning Guide from our website (mellanox.com) and validate BIOS & OS tuning for maximum performance.

  3. From our community site, follow document:

HowTo Configure RSS on ConnectX-3 Pro for Windows 2012 server https://community.mellanox.com/s/article/howto-configure-rss-on-connectx-3-pro-for-windows-2012-server DOC-2312

Also applicable in your environment.

Sophie.