Error in ib_send_bw and ib_write_lat using MLNX OFED 4.5.1.0.1.0

I have a setup of two nodes each with a ConnectX-5 NIC and these NICs are directly connected to each other. One node has kernel version 4.17.0 and other has 4.19.10 and both run Fedora 28. Both of the nodes have MLNX OFED 4.5.1.0.1.0 installed on them which was built using the mlnx_add_kernel_support.sh. I am following this mellanox post

https://community.mellanox.com/s/article/howto-setup-rdma-connection-using-inbox-driver--rhel--ubuntu-x

to verify the connectivity between the nodes. The udaddy, rdma_server/rdma_client, rping, ucmatose commands work as expected. However running the ib_send_bw (ib_send_bw -d mlx5_0 -i 1 -F --report_gbits 12.12.12.2) fails with the following error

Couldn’t connect to 12.12.12.2:18515

Unable to open file descriptor for socket connection Unable to init the socket connection

Similar output is observed for ib_write_lat and ibv_rc_pingpong. What can be the reasons behind this and how to resolve it ???

Hi Sumit,

Validate firewall is disabled on both hosts.

Port 18515 is the default port used by the utilities, make sure it is not already is used. You can use -p to listen on/connect to a different port.

Try using "ib_send_bw -d mlx5_0 -i 1 -R --report_gbits" on both side and omit -F.

Can you ping between both servers?

Can you ibping between both servers?

Sophie.

Hello Sophie,

Disabling the firewall on both hosts solved the problem. The setup is working now.

Thank you,

Sumit

Good:-)

Sophie.