Local ib_read_lat fails on Mellanox Technologies MT27700 Family [ConnectX-4]

Infrastructure & Networking Adapters and Cables InfiniBand/VPI Adapter Cards
1

Dear experts,

running ib_read_lat fails with this error message:

[root@ib_machin ~]# ib_read_lat -F localhost

                RDMA_Read Latency Test

Dual-port       : OFF          Device         : mlx5_0
Number of qps   : 1            Transport type : IB
Connection type : RC           Using SRQ      : OFF
PCIe relax order: ON           Lock-free      : OFF
ibv_wr* API     : ON           Using DDP      : OFF
TX depth        : 1
Mtu             : 4096[B]
Link type       : IB
Outstand reads  : 16
rdma_cm QPs     : OFF
Data ex. method : Ethernet

local address: LID 0x188a QPN 0x18b21 PSN 0xd10e8a OUT 0x10 RKey 0x001500 VAddr 0x00563542210000
remote address: LID 0x188a QPN 0x18b22 PSN 0x28d992 OUT 0x10 RKey 0x001600 VAddr 0x00560528be7000


 
    t_min[usec]    t_max[usec]  t_typical[usec]    t_avg[usec]    t_stdev[usec]   99% percentile[usec]   99.9% percentile[usec]
Completion with error at client
Failed status 12: wr_id 0 syndrom 0x81
scnt=1, ccnt=1

It’s an OFED-internal-25.04-0.6.0: system with a Mellanox Technologies MT27700 Family [ConnectX-4].

Any ideas how to look into this?

Cheers, Peter

2

In the meantime, a power off, sleep, power on did fix the issue.