After reading this post:
And applying some of the tweak, we’re still not able to get more that 11 Gbps up and 17.7 Gbps down per link.
We absolutely need more that 20Gbps per link BOTH way.
Is there any fix for this comming at some point, or should we look for a different platform/vendor
this is a known issue which is not yet fixed. Still under investigation.
Any ETA on a fix as this is critical to our application, and if not resolved soon, we will have to look at something else.
No answer.. which means no ETA, and may be we should look at something else then.
Please wait for the next JetPack 7.2 GA release.
@rodolphe.pineau Once the Thor ports are already coming up at the 25GbE link rate, the remaining ceiling appears to be in the MGBE software path, not in the QSFP breakout, MikroTik port mode, or the Mellanox side. That matches what Wayne said in the earlier thread and what NVIDIA staff are saying here now: it is a known issue, and the next public thing to wait for is JetPack 7.2 GA.
We hit the same shape on the earlier thread you linked. What I actually observed there was that the documented kernel/BPMP DTB changes were necessary to get the ports into the right link mode, and Wayne’s jetson_clocks + threaded MGBE/NAPI suggestion helped after link establishment. We also found one local duplicate-MAC issue on a port, which is worth checking with ip -s -a addr if one lane behaves differently from the others. But none of that is a claim-safe workaround for your requirement of sustained bidirectional application throughput above your threshold.
The part I would not over-read from my prior post is the MACsec direction. We had identified it as a possible next thing to investigate, but I did not validate “disable MACsec” as a production fix, and there were other Orin/Jetson reports that made that path look non-obvious rather than clean. I would not treat that as the missing step for a product schedule.
So my read, as another user who ran into the same class of issue: if your DTB/BPMP patching is correct, all links report the expected link mode, threaded MGBE is enabled, clocks are locked, MAC addresses are unique, and both directions are still capped, I would treat this as blocked on NVIDIA’s JetPack-side fix rather than a remaining user-space tuning problem.
I agree. The issue is on NVIDIA’s side and all we can do is wait for Jetpack 7.2GA and hope it’s fixed.
Same read here. Worth pinging this thread once anyone is on 7.2 GA with actual measurements — we’ll be running the same MGBE bandwidth test on our side and can post numbers either way once the update lands.