Can we fix the number of GPU Cores used during a TensorRT inference execution? & is (Multi-instance GPU) MIG Available on Jetson AGX Thor

Robotics & Edge Computing Jetson Systems Jetson Thor
,
6

Hello @neethujoseph02,

Currently, seems that there is no built-in tool on Jetson Thor that reports the memory usage of individual MIG instances. tegrastats only reports aggregate system/GPU usage, and nvidia-smi does not provide a per-MIG memory breakdown on Jetson Thor.

Regarding the SM count, the 12 + 6 split reported by nvidia-smi is a known issue. NVIDIA has acknowledged that the output is incorrect:

“There is a known issue 6162096 — Output of nvidia-smi is incorrect when using MIG. This gives you incorrect information of the SM split being 12 and 6.”

The discussion is available here.

In summary, the remaining 2 SMs are not lost; the discrepancy is due to the current nvidia-smi reporting issue rather than the actual hardware partitioning.

Regards,
Daniel R.
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com