Running Nemotron 3 Super 120B on DGX Spark GB10— 72 hours continuous, 19 tok/s

AI & Data Science NVIDIA Nemotron
,
1

Built financial intelligence platform on DGX Spark.
Nemotron 120B running 72+ hours without interruption.
Q4_K_M quantization via llama.cpp native GB10 compile.

pip install yuclaw

4

Technical details for anyone trying to run
Nemotron on DGX Spark GB10:

The challenge: GB10 is CUDA compute 12.1
but PyTorch only supports up to 12.0.

What failed:

  • vllm crashed — sm_121a incompatibility
  • BF16 225GB too large for 128GB unified memory

What worked:

  1. Build llama.cpp with native CUDA arch:
    cmake -B build -DGGML_CUDA=ON
    -DCMAKE_CUDA_ARCHITECTURES=“native”

  2. Convert FP8 → GGUF BF16 (11 minutes)

  3. Quantize BF16 → Q4_K_M (~65GB fits in GPU)

  4. Run with tmux for 72hr stability

Result: 19 tok/s, 81GB VRAM, stable.

Full implementation: YuClawLab · GitHub