I would like to guesstimate performance of YOLOv7 model on a Jetson Orin system. I’m benchmarking it on a PC with RTX 3050 GPU, which is based on the same Ampere architecture. Are there any significant architectural differences between RTX 30xx line and Jetson Orin, other than CUDA cores number and clock frequency, which would influence inference performance on models like YOLOv7?
The memory in an RTX 3050 is within the discrete GPU (dGPU). I think this is probably GDDR6. The dGPU talks via the PCI bus.
The Jetson uses an integrated GPU (iGPU), and is directly integrated with the memory controller without any of its own memory (it shares with system memory…much slower than GDDR6 and in competition with other users of that memory, although it does not need to wait for the PCI bus for I/O).
Most likely the PC version has many releases of GPU-related software available (e.g., CUDA software). The version the Orin uses is tied to the L4T release it is flashed with (JetPack/SDK Manager is the front end software which a host PC uses to flash the Jetson; L4T is what actually gets flashed, and is Ubuntu 20.04 plus NVIDIA drivers).
Jetsons do have various power modes, and so for any testing you’d want to make sure all CPU cores are enabled, and the clocks maximized (a combination of the nvpmodel and running jetson_clocks to maximize within that power model).
Good point about GPU memory bandwidth. The datasheets claim 32GB 256-bit LPDDR5 at 204.8 GB/s for Jetson AGX Orin and 8GB 128-bit GDDR6 at 224 GB/s for RTX 3050. A small bandwidth penalty for Orin, but much larger fast memory.
Isn’t the most current CUDA version available for Jetson Orin too?
The L4T versions and what they are available for are listed here:
https://developer.nvidia.com/linux-tegra
JetPack/SDKM versions (which are what flash L4T…the versions are locked together) are listed here:
https://developer.nvidia.com/embedded/jetpack-archive
If you go to either, and find the most recent for either L4T or JetPack, then it will lead to a URL with both so far as specifications and downloads. The URL for that particular release tells you which CUDA release is available when you dig in to those documents (currently L4T R35.1).
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.