100G jetpack 7.2

We are using a custom carrier board and followed the official testing procedure described in the NVIDIA documentation below to run the 24-stream iperf performance test:

We observed approximately 85 Gbps for UDP throughput and around 67 Gbps for TCP throughput.

To verify whether this was a hardware issue, I also performed the same test on two NVIDIA developer kits running the 7.2 release, and obtained nearly identical results to our custom board.

However, according to the performance numbers provided in the official documentation, the aggregated throughput should be approximately 103 Gbps for UDP and 93 Gbps for TCP.

I have followed all of the configuration and test steps described in the documentation. Is there anything else that I might have missed, or are there any additional factors that could affect the final throughput?

I also noticed that in the official UDP results, one stream reaches 9.42 Gbps, which is significantly higher than the average throughput of the other streams. Similarly, in the TCP results, one stream reaches 6.17 Gbps, while most of the other streams are around 2.5 Gbps. Could you explain why these particular streams achieve much higher throughput?

In addition, I observed that the hardware interrupts are still almost entirely handled by CPU0. Could this have an impact on the overall network performance?

custom board:

thor devkit:

official doc:

interrupt:

this is under internal check. will update later.

Hello,WayneWWW

I also noticed that, even without making any modifications, the throughput suddenly increased. In some cases, the aggregated throughput even exceeded the line rate, especially with brief spikes. I suspect this is due to the way iperf calculates and reports throughput. However, increasing the test duration with the -t option makes the measured throughput closer to the actual value.

small time :

60s:

After applying optimizations such as IRQ distribution, the reported throughput can exceed 100 Gbps. Hmm, I don’t think that result is very accurate.