How to monitor one-sided RDMA bandwidth at QP level?

Infrastructure & Networking Software And Drivers Mellanox OFED
1

I want to monitor one-sided RDMA bandwidth at receiver side (whose CPU is bypassed) at QP-level.
I noticed that mlx5 based RNICs provide port counters and hardware counters ( Understanding mlx5 Linux Counters and Status Parameters (nvidia.com)). But port_rcv_data only provides port-level size of data transferred, and rx_read/write_requests only provide QP-level number of requests received.
My question is that is there any method to monitor one-sided RDMA bandwidth at receiver side (whose CPU is bypassed) at QP-level?

2

So far, no QP-level counters. But it can be easy static from app API level.

3

There is a mechanism to do that via “rdma statistics” command.
Please review these links:
https://man7.org/linux/man-pages/man8/rdma-statistic.8.html

Regards,
Yaniv

4

Thanks for your reply. Yes, it is trivial to static in the sender’s app level. However, I’m wondering how to static at the receiver side, which cannot be done at app level since its CPU is bypassed.

5

Thanks for your information. I see it can provide QP-level number of read/write requests. But it is not enough to calculate bandwidth since data transferred in each request is still unknown.

6

Assuming you know the msg size for this QP (if it is constant) you can estimate what you are looking for.

7

Sure. It can be a coarse-grained solution to identify which QP connection is “hot”.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.