I am trying to data transfer between two CX5 ports usinf ib_write_bw. but it is failing.
[root@ctech123 64]# ibdev2netdev
mlx5_0 port 1 ==> enp94s0f0 (Up)
mlx5_1 port 1 ==> enp94s0f1 (Up)
[root@ctech123 64]#
mlx5_0 is live port .
mlx5_0 and mlx5_1 are connected by cable.
Server
[root@ctech123 64]# ib_send_bw -F -d mlx5_0 -D 66 -p 8005 --report_gbits -b
- Waiting for client to connect… *
Send Bidirectional BW Test
Dual-port : OFF Device : mlx5_0
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
TX depth : 128
RX depth : 512
CQ Moderation : 100
Mtu : 1024[B]
Link type : Ethernet
GID index : 4
Max inline data : 0[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
local address: LID 0000 QPN 0x0115 PSN 0x99e04e
GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:134:145:61
remote address: LID 0000 QPN 0x091c PSN 0x39922e
GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:134:01:01
ethernet_read_keys: Couldn’t read remote address
Unable to read to socket/rdam_cm
Failed to exchange data between server and clients
[root@ctech123 64]#
Client Side
[root@ctech123 64]# ib_send_bw -F -d mlx5_1 -D 66 --report_gbits -p 8005 10.134.1.1 -b -m 4096
Requested mtu is higher than active mtu
Changing to active mtu - 3
Send Bidirectional BW Test
Dual-port : OFF Device : mlx5_1
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
TX depth : 128
RX depth : 512
CQ Moderation : 100
Mtu : 1024[B]
Link type : Ethernet
GID index : 2
Max inline data : 0[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
local address: LID 0000 QPN 0x091c PSN 0x39922e
GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:134:01:01
remote address: LID 0000 QPN 0x0115 PSN 0x99e04e
GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:134:145:61
Failed to modify QP 2332 to RTR
Unable to Connect the HCA’s through the link
[root@ctech123 64]#