dzhao05
November 14, 2025, 6:13pm
1
If you are reporting a bug or error , consider submitting a Support Bundle to aiworkbench-ea@nvidia.com . This will help us solve your issue more quickly.
Please describe your issue or request: (tick the boxes after creating this topic):
Please tick the appropriate box to help us categorize your post Bug or Error
Hi,
I’m running a DGX Spark with a GB10 GPU (128GB VRAM).
Inside the AI Workbench environment, nvidia-smi shows:
FB Memory Usage: N/A
Total/Used/Reserved memory: N/A
MIG Mode: N/A
Effective VRAM only ~56GB
vLLM fails to start because it requires >100GB free memory
On the host machine (outside Workbench), the GPU shows the full 128GB normally.
This suggests that Workbench is using a virtual GPU (vGPU / SR-IOV) profile instead of giving full GPU passthrough.
Support asked me to post this question here.
How can I configure AI Workbench / DGX Spark to expose the full 128GB GPU memory to containers?
NVES
November 16, 2025, 2:48am
2
can you please post the docker command used to launch the ai workbench container?
dzhao05
November 18, 2025, 7:04pm
3
Where to get the relevant instructions? thanks
Hi, can you provide a screenshot of your nvidia-smi output using AI Workbench? Due to the DGX Spark having unified memory, nvidia-smi cannot report on GPU memory usage so it is unclear what number you are seeing that only shows 56GB
dzhao05
November 19, 2025, 8:26am
5
Could you please simply tell me the correct command to see that the available is 128G? thanks.
It is unclear what you are seeing in the first place. nvidia-smi cannot show memory usage due to the unified memory architecture so we do not understand where you are seeing this 56GB number so we are asking for more details
This is invalid. I can run models using vLLM on docker with high-utilization. I suggest you start using llama.cpp over ollama, it’s much more performant.
Sample with vLLM: Running nvidia/Nemotron-Nano-VL-12B-V2-NVFP4-QAD on your spark
Performance numbers, using containers with high-utilization with 1 or 2 sparks:
With properly setup dual Spark cluster you can expect almost 2x performance gain for dense models (slow ones) and less gain for sparse ones. Prompt processing performance scales better than inference.
Here is a compilation of my results - some of these need retesting as I was running with old config, but you can get an idea. That’s using VLLM and two Sparks connected via a single QSFP112 cable.
Model name
Cluster (t/s)
Single (t/s)
Comment
Qwen/Qwen3-VL-32B-Instruct-FP8
12.00
7.00
c…
@dzhao05 Are you still running into this issue? If so can you give more details?