I’m trying to clarify the transport expectations for a Holoscan Sensor Bridge-compatible sensor when targeting Jetson AGX Thor.
My current understanding is:
- On AGX Thor, the documented accelerated path uses MGBE / CoE / SIPL
- On ConnectX-based systems (for example IGX), the accelerated path can use RoCEv2 / RDMA
- Thor MGBE is described separately from ConnectX, which makes it unclear whether the distinction is only on the host receiver side or also on the sensor-side protocol
What I need to understand specifically is this:
- CoE protocol, or
- RoCEv2 protocol
will both of these work when connecting directly to the MGBE MAC interface on AGX Thor?
More concretely:
- Is AGX Thor MGBE capable of receiving a RoCEv2-speaking sensor endpoint directly, or is CoE the only supported transport for direct MGBE connection on Thor?
- If RoCEv2 is not supported on direct Thor MGBE, is that because of:
- a PHY/MAC/link-layer incompatibility,
- a transport/protocol mismatch above Ethernet,
- or simply lack of software / driver / receiver support in the current Holoscan stack?
- On the sensor side, should developers think of CoE sensors and RoCE sensors as meaningfully different endpoint implementations, or is the distinction primarily on the host-side receive path?
- If a sensor is implemented as a generic Ethernet / FPGA endpoint, is the correct statement:
- “To work with Thor direct MGBE, it must speak CoE”
- while “To work with ConnectX NIC accelerated receive, it must speak RoCEv2”?
I’m trying to avoid designing a sensor endpoint that is only compatible with one class of host if that is not actually required by the architecture.
Some sites that might help. The first is specifically about HSB and Thor:
https://docs.nvidia.com/jetson/archives/r38.4/DeveloperGuide/SD/CameraDevelopment/CoECameraDevelopment/CameraSoftwareDevelopmentSolutionThor.html
https://docs.nvidia.com/holoscan/sensor-bridge/2.5.0/index.html
https://github.com/nvidia-holoscan
https://github.com/nvidia-holoscan/holoscan-sensor-bridge
https://github.com/nvidia-holoscan/holohub
Thanks! These pages were all part of my original review.
Unfortunately none of them make it clear whether the MGBE / CoE connections on the AGX Thor SoM can be directly connected to a sensor that exclusively supports RoCEv2, or if a ConnectX SmartNIC is required to enable RoCEv2 intead.
Hi,
I got this from document:
This says which is the setup needed for both cases. Also checking the Jetson Thor camera development documentation related, we can only found support for CoE,
NVIDIA ConnectX-7 (CX7) SmartNIC is a high-performance networking card designed for datacenter, AI, and HPC workloads, offering up to 400Gb/s speeds, while MGBE SmartNIC generally refers to Gigabit Ethernet interfaces used in edge AI systems (like Jetson Thor) for lower-speed sensor connectivity.
So I would say RoCEv2 is not supported by Jetson Thor. The CoE is the supported implementation on this system.
Enrique Ramirez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com
Thanks Enrique! I am seeing the same on my end. I am hoping that someone from NVIDIA can clarify once and for all what is supported / not supported on those MGBE SmartNIC devices. It doesn’t seem to be entirely clear if they support RoCEv2 currently, although I agree with you, it appears that they do not.
Holoscan Sensor Bridge FPGA IP core has two modes of data plane operation: RoCEv2 or CoE. When connected to Thor using SIPL or HSB-direct APIs, the Thor SoC will auto-negotiate a CoE connection.
Thor MGBE is not ConnectX-based. It does not support RoCE, DPDK, Rivermax, or other technologies found in our ConnectX-based products. It offers hardware acceleration for Holoscan Sensor Bridge ingress only via the CoE mode.