I am facing an issue with my device, I am unable to get maximum bandwidth throughput for file transfer or any other RDMA operations. I get buffer limit issue which I don’t understand because rdma are supposed to have the maximum bandwidth utility but I am only getting 8.13 mbit/sec while rdma transfer supposed to have 9.83gbit/sec or maximum? can you please confirm if there is any issue with the device or provide me with the documentation regarding this specific device
Hello baka_laowai,
Thank you for posting your inquiry on the NVIDIA Developer Forum - Infrastructure and Networking - Section.
For the adapters to reach max. throughput for RDMA, please make sure all nodes are tuned to max. performance.
Please review the following resources to understand Performance Tuning and how to apply it for your use case.
- https://support.mellanox.com/s/article/understanding-bios-configuration-for-performance-tuning
- https://support.mellanox.com/s/article/performance-tuning-for-mellanox-adapters
- https://support.mellanox.com/s/article/how-to-set-cpu-scaling-governor-to-max-performance--scaling-governor-x
- https://support.mellanox.com/s/article/understanding-numa-node-for-performance-benchmarks
Make sure your are running the latest f/w and driver for your adapter → Linux InfiniBand Drivers
Thank you and regards,
~NVIDIA Networking Technical Support
I have changed the settings in CPU frequency, I changed the scaling governor to “performance” instead of “power-safe” to utilize the maximum CPU frequency but still I am unable to see high throughput when transferring file. If there is some application to test the benchmark, can you share it ?
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.