BlueField3 Msg Rate Test

[1054919371]

I was trying to test the msg rate.
With netperf, use 64 points to test and i got this. It’s much far away from the MAX.
What can I do now?

[ssimcoejr]

Hello 1054919371,

Thank you for posting your inquiry to the NVIDIA Developer Forums.

Some troubleshooting/tuning recommendations follow - if these do not resolve your issue, and you have valid support entitlement, we highly recommend engaging NVEX Support for further assistance on this issue.

1. System Tuning

• Use tuned-adm profile network-throughput to set optimal network parameters

• Disable IRQ balancing: systemctl stop irqbalance.service

• Optimize CPU governor for performance

2. Network Interface Optimization

• Increase interface queue size: ethtool -G <interface> rx 8192 tx 8192

• Disable adaptive coalescing: ethtool -C <interface> adaptive-rx off adaptive-tx off

• Optimize interrupt moderation: ethtool -C <interface> rx-usecs 0 rx-frames 0

• Set appropriate queue count: ethtool -L <interface> combined 31

3. IRQ Affinity Optimization

• Use set_irq_affinity_cpulist.sh to assign IRQs to specific CPUs

4. RSS Configuration for BlueField-3

• Implement proper RSS rules to distribute traffic across multiple queues

• Ensure 1:1 ratio between streams and RX queues

• Use multiple IP addresses for better queue utilization

• Apply TC rules with set_tc_rules.sh p0 <interface_name>

5. Netperf Testing Methodology

• Start with a single stream baseline test, then scale up

• Use: netperf -H <server_ip> -t UDP_STREAM -l 60 -- -m 64

• For multiple streams, launch parallel netperf instances

• Test with different message sizes (8, 64, 128, 512 bytes)

Best regards,
NVIDIA Enterprise Experience