Hi all, i have configured cluster S2D , two nodes with rdma adapters (HP 546SFP+ ) connected to Switch Dell PowerConnect 8024 with lastest firmware.
Cluster have two networks : 1 routable for VMs, 2 non routable for Storage RDMA
I use this PS code for configure RDMA on nodes…
Install-WindowsFeature Data-Center-Bridging Set-NetQosDcbxSetting -Willing $False Enable-NetAdapterRdma –Name RDMA-NIC1 Remove-NetQosTrafficClass -confirm:$False Remove-NetQosPolicy -confirm:$False New-NetQosPolicy "SMBDIRECT" -netDirectPortMatchCondition 445 -PriorityValue8021Action 4 New-NetQosPolicy "DEFAULT" -default -PriorityValue8021Action 1 Enable-NetQosFlowControl -Priority 4 Disable-NetQosFlowControl 0,1,2,3,5,6,7 Enable-NetAdapterQos -InterfaceAlias "RDMA-NIC1" New-NetQoSTrafficClass "SMBDirect" -Priority 4 -Bandwidth 90 -Algorithm ETS
Flow control disabled on NIC
GlobalPause disabled on NIC
VlanID set to 666
Switch config:
Jumbo Frames enabled
flowcontrol globally disabled(global pause)
Cut-through mode is configured as enabled
Ports config:
interface Te1/0/20 datacenter-bridging priority-flow-control mode on priority-flow-control priority 4 no-drop exit spanning-tree portfast mtu 9216 switchport mode trunk exit ``interface Te1/0/10 datacenter-bridging priority-flow-control mode on priority-flow-control priority 4 no-drop exit spanning-tree portfast mtu 9216 switchport mode trunk exit
After configuration i see RDMA activity counters greater than zero… RDMA works, but i see Mellanox Diagnostics counters have many errors… screen shot attached
nd_send_bw test results:
C:\Program Files\Mellanox\MLNX_VPI\IB\Tools>nd_send_bw -C 10.100.100.2
#qp #bytes #iterations MR [Mmps] Gb/s CPU Util.
0 65536 100000 0.017 9.13 100.00
Test finished. Releasing resources…
Can i fix errors or this is correct RDMA work? Can verify RDMA works correctly?
thx for help…
