Maximum model size to build TRT-LLM Engine on DGX Spark?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
,
1

I’m trying to convert nvidia/Llama-3_3-Nemotron-Super-49B-v1_5-FP8 into a TRTLLM Engine.

I pull the 1.10rc3 container optimized for the spark. and it starts to load weights. It gets to

[TensorRT-LLM] TensorRT-LLM version: 1.1.0rc3
[TensorRT-LLM][INFO] Refreshed the MPI local session
You are using a model of type nemotron-nas to instantiate a model of type nemotron_nas. This is not supported for all configurations of models and can yield errors.
Loading safetensors weights in parallel: 100%|██████████| 11/11 [00:00<00:00, 42.53it/s]
Loading weights concurrently: 100%|██████████| 1219/1219 [03:20<00:00, 6.07it/s]
Model init total – 216.25s
[TensorRT-LLM][INFO] Number of tokens per block: 32.
[TensorRT-LLM][INFO] [MemUsageChange] Allocated 0.77 GiB for max tokens in paged KV cache (8256).
2025-10-26 18:21:43,982 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-10-26 18:21:57,794 - INFO - flashinfer.jit: Finished loading JIT ops: norm
[TensorRT-LLM][WARNING] Attention workspace size is not enough, increase the size from 0 bytes to 100732416 bytes
[TensorRT-LLM][WARNING] CheckTactic failed with status: 8 and heuristic status: 8 with workspace size: 28.

(That checktactic error goes on repeatedly.)

Am I just running out of memory because that’s too large of a model to build an engine on GB10? Or is there a problem with my code and I just need to keep debugging?

Thanks. (And does it matter that 1.1.0rc3 runs CUDA 12.9 even though it’s the DGX Spark container
nvcr.io/nvidia/tensorrt-llm/release:spark-single-gpu-dev

2

The DGX Spark supports AI models up to 200 billion parameters, so a 49B model should not be the issue here unless you are using system memory to run other workloads at the same time

3

Thanks. My understanding is that while I am able to deploy the model using GGUFs easily (where the 200B NVFP4 metric comes from), if I am building a TRT-LLM engine, my understanding is that it can require much more VRAM than the model itself? So even though Nemotron 49B v1.5 is 53GB in Q8, it needs more than 128GB unified RAM to build a NVFP4 or Q8 engine?

Nemotron Nano 9B v2 works, but I just want to know if NVIDIA has some guidelines for TRT-LLM engine building on DGX Spark in terms of model size that leaves enough overhead.

4

I did not see that you wanted to build a trt-llm engine. It is very possible you do not have enough memory to build an engine, I’m not sure on the memory requirements for that process.
However, you can use the TensorRT optimizer to create a NVFP4 quantized version of the model and deploy it with TRT-LLM. You can find an example in our playbooks: Try NVIDIA NIM APIs