Hi,
cases where the behaviour we measured does not match what we expected from
the documentation, and I would like to know which of them are intended so I can plan around them correctly.
System
Jetson AGX Thor
L4T R39.2.0 (/etc/nv_tegra_release) -> JetPack 7.2
nvidia-l4t-core 39.2.0-20260601141651
Driver 595.78, CUDA 13.2
Ubuntu 24.04.4
20 SMs, 125748 MB, num_DLA_cores = 0
MIG mode: Enabled, persistence mode Enabled
The full profile table, on a board with no instances created:
$ sudo nvidia-smi mig -i 0 -lgip
| GPU Name ID Instances Memory P2P SM DEC ENC |
| Free/Total GiB CE JPEG OFA |
| 0 MIG 1g.0gb+me 78 1/1 0.00 No 6 1 1 |
| 0 MIG 1g.0gb 80 1/1 0.00 No 6 1 1 |
| 0 MIG 1g.0gb+gfx 81 1/1 0.00 No 6 1 1 |
| 0 MIG 2g.0gb 82 1/1 0.00 No 12 1 1 |
| 0 MIG 2g.0gb+gfx 83 1/1 0.00 No 12 1 1 |
| 0 MIG 3g.0gb 0 1/1 0.00 No 20 2 2 |
| 0 MIG 3g.0gb+gfx 32 1/1 0.00 No 20 2 2 |
Q1 - granularity is 6 / 12 / 20 SM, with no 8 SM profile
The documentation of running both SONIC and GR00TN1.7 clearly state that SMs should be split 12 + 8. However there is no a single instance that supports 8 SMs under -lgip.
Q2 - multi_processor_count disagrees with the SM column, and only for the plain variants
Reading cudaGetDeviceProperties from inside each instance:
| Profile | -lgip SM column | multi_processor_count |
|---|---|---|
| `1g.0gb` (80) | 6 | **8** |
| `1g.0gb+gfx` (81) | 6 | 6 |
| `2g.0gb` (82) | 12 | **14** |
| `2g.0gb+gfx` (83) | 12 | 12 |
So the plain variants report more SMs than advertised, while the +gfx variants
report exactly the advertised figure. My guess is that the table is a per-GPC nominal
and the plain variants additionally pick up SMs the graphics engine would otherwise
occupy, but that is a guess.
Which number is authoritative when sizing a TensorRT engine?
We ask because getting this wrong does not produce a clean failure. A plan built against the full 20 SMs, deserialized on a smaller instance, emits:
The current device only supports 14 multiprocessors but the engine file requires 20.
Deadlocking is likely.
and then deadlocks exactly as advertised. Rebuilding the plan pinned to the target
instance fixes it, so we can work with either number we just need to know which one
to build against, and whether it is stable across driver updates.
Q3 - a two-instance layout is refused unless exactly one instance is +gfx
With three placement slots, 2g + 1g should be the natural split. It works only in
certain combinations. Starting from an empty board:
$ sudo nvidia-smi mig -i 0 -cgi 82 -C # 2g.0gb, lands at placement 0:2
Successfully created GPU instance ID 1 on GPU 0 using profile MIG 2g.0gb (ID 82)
$ sudo nvidia-smi mig -i 0 -cgi 80 -C # 1g.0gb into the free slot 2
Unable to create a GPU instance on GPU 0 using profile 80: Insufficient Resources
Failed to create GPU instances: Insufficient Resources
$ sudo nvidia-smi mig -i 0 -cgi 78 -C # 1g.0gb+me, same free slot
Unable to create a GPU instance on GPU 0 using profile 78: Insufficient Resources
$ sudo nvidia-smi mig -i 0 -cgi 81 -C # 1g.0gb+gfx, same free slot
Successfully created GPU instance ID 2 on GPU 0 using profile MIG 1g.0gb+gfx (ID 81)
At the point of the two failures, -lgip showed 1g.0gb+gfx as 1/1 while both
1g.0gb and 1g.0gb+me showed 0/1, with the placement slot genuinely free.
The mirror image also works: creating plain 1g.0gb first and then 2g.0gb+gfx succeeds.
Is the rule that the board has one graphics-capable GPC which must be claimed by
whichever instance covers it?
“Insufficient Resources” pointing at a visibly free placement slot is a hard error to diagnose, and a message naming the graphics constraint would have saved us time.
Please any further guidance and answering these questions can be very helpful.