mellanox connectX3 adapter failed to reach the rational speed

Hello,

We are using the mellanox connectx3 adapter(Link Speed:40Gps) on windows10 PCs. We also use the proper infiniband cable for this kind of adapter which can provide the rational infiniband speed .

However, we still failed to reach the rational speed. I use the nd_read_bw to test the speed , it was just about 6Gps. It’s further out of our expectation.

So l use the command “vstat” to show some info about my PC:

This is vstat result on the same host:11.4.12.11

C:\Windows\system32>vstat

hca_idx=0

uplink={BUS=PCI_E Gen2, SPEED=5.0 Gbps, WIDTH=x2, CAPS=8.0*x8}

MSI-X={ENABLED=1, SUPPORTED=128, GRANTED=10, ALL_MASKED=N}

vendor_id=0x02c9

vendor_part_id=4099

hw_ver=0x0

fw_ver=2.42.5000

PSID=MT_1100110028

node_guid=f452:1403:0047:63f0

num_phys_ports=1

port=1

port_guid=f452:1403:0047:63f1

port_state=PORT_ACTIVE (4)

link_speed=10.00 Gbps

link_width=4x (2)

rate=40.00 Gbps

real_rate=32.00 Gbps (QDR)

port_phys_state=LINK_UP (5)

active_speed=10.00 Gbps

sm_lid=0x0004

port_lid=0x0004

port_lmc=0x0

transport=IB

max_mtu=4096 (5)

active_mtu=4096 (5)

GID[0]=fe80:0000:0000:0000:f452:1403:0047:63f1

This is test result on the same host:11.4.12.11

C:\Windows\system32>nd_read_bw -S 11.4.12.11

Listening for incoming connection request…

Connection accepted.

C:\Windows\system32>nd_read_bw -S 11.4.12.11

Listening for incoming connection request…

Connection accepted.

C:\Windows\system32>nd_read_bw -C 11.4.12.11

#qp #bytes #iterations MR [Mmps] Gb/s CPU Util.

0 65536 100000 0.011 5.95 100.00

Test finished. Releasing resources…

C:\Windows\system32>nd_read_bw -C 11.4.12.11

#qp #bytes #iterations MR [Mmps] Gb/s CPU Util.

0 65536 100000 0.011 5.94 100.00

Test finished. Releasing resources…

Can someone help me solve the problem why we can’t reach the rational speed?

We have already changed the appropriate cable as well as change the slot to inappropriate PCI-E slot. And it worked, we have already reached the rational speed.