In case this helps anyone, I had to render a few hundred images and audio files, so invested some time to optimize things. I’m blown away by the improvements from a first install to optimized. Crazy results!!
PSA: ComfyUI
--use-sage-attentionon DGX Spark may be silently running PyTorch attention (missing python3.12-dev → 20× slowdown)
TL;DR: SageAttention was silently inactive on both of our DGX Sparks. Its failure mode is graceful fallback to PyTorch attention, so everything still renders — but Triton re-attempts a doomed gcc compile every sampling step, starving the GPU. Fix: sudo apt install python3.12-dev. SDXL 1024² went from ~140 s to 6–8 s; 4096² renders in 17–18 s.
Here’s what I’m running:
| component | version |
|---|---|
| Hardware | DGX Spark (GB10, sm_121a, 121.7 GiB unified) ×2 |
| OS / kernel | Ubuntu 24.04.4 (DGX OS), 6.17.0-1014-nvidia, driver 580.142 |
| Install type | bare-metal venv (per NVIDIA’s own Spark ComfyUI playbook — no container) |
| Python | 3.12.3 (system), venv per node |
| torch | 2.12.1+cu130 (aarch64 wheels, --index-url .../whl/cu130) |
| ComfyUI | 0.27.0 (b08debce) |
| sageattention | 1.0.6 (plain pip install sageattention — works on GB10) |
| triton | 3.7.1 |
| launch flags | --use-sage-attention --bf16-unet --bf16-vae --bf16-text-enc --reserve-vram <N> |
(The --reserve-vram is because the Sparks co-serve vLLM endpoints; note it subtracts from CUDA-free memory, not total.)
Results (SDXL, bf16, Sage active, per prompt end-to-end)
| render | before (worst case) | after |
|---|---|---|
| 1024×1024 (SDXL+ControlNet) | ~140 s | 6–8 s |
| 4096×4096 | — | 17–18 s |
The 4096² number is the fun one: attention scales ~quadratically with sequence length, so high-res is exactly where Sage pays off — 121GB memory means the 4096² latents just fit, no tiling, no OOM, while the same boxes keep serving LLMs (vLLM) alongside.
Failure mode
SageAttention 1.x runs through Triton, which JIT-compiles a small C shim with the host gcc at runtime. DGX OS doesn’t ship the Python dev headers, so the compile fails:
/tmp/tmpXXXX/cuda_utils.c:9:10: fatal error: Python.h: No such file or directory
[ERROR] Error running sage attention: Command '['/usr/bin/gcc', ...]' returned non-zero exit status 1., using pytorch attention instead.
The failure is not cached — ComfyUI retries Sage on every attention call, so every step spawns a fresh gcc that fails again. Symptoms: GPU utilization stuck at 8–15%, one busy CPU core, suspiciously low GPU temps, 1024² SDXL+ControlNet ≈ 140 s. The fallback is silent, so you can run like this indefinitely.
Fix:
sudo apt install -y python3.12-dev
No restart needed — the next attention call compiles, caches, and Sage engages mid-queue (a running queue dropped from ~140 s to ~25 s per prompt the moment apt finished; a ComfyUI restart to reload models fully resident got us the rest of the way).
Quick check
grep -c 'Error running sage attention' <your comfyui log>
Growing count = you are not running Sage, whatever your launch flags say.