ComfyUI setup optimized for DGX Spark

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
,
1

Hello everyone,

I’m sharing here my custom docker setup for ComfyUI. I’m actively using it and trying to find improvements as I go. Tested various other setups but none of them performed very well or the way i liked, each with their own issues.

Here you can find a github repo with the docker image: GitHub - luix93/DGX-Spark-ComfyUI: An optimized setup for running ComfyUI on DGX Spark · GitHub

Main features:

  • CUDA 13.1 base — full nvcc support for GB10 (sm_121), enabling CUDA extension compilation

  • PyTorch cu130 — prebuilt ARM64 wheels from PyTorch’s cu130 index

  • SageAttention 2 — compiled from source directly against sm_121 for full hardware attention acceleration

  • Comfy Kitchen (comfy_kitchen) — NVFP4 quantization support for Blackwell

  • Unified-memory optimized flags — carefully tuned COMFYUI_FLAGS that avoid fighting the Grace-Blackwell memory fabric

  • Double-VRAM bug fix — patches comfy/utils.py to set copy=False in tensor.to(), fixing the double memory usage on unified memory systems with --disable-mmap

  • Disabled dynamic vram — uses --disable-dynamic-vram as it doesn’t work properly on the Spark, if models fit in memory they won’t be unloaded, faster prompt changes to final image/video

  • ComfyUI-Manager — auto-installed at container startup into the mounted custom_nodes volume

  • ComfyUIMini — lightweight mobile/tablet UI proxying to the ComfyUI backend (optional second service)

  • Health checks — both services expose health check endpoints for reliable depends_on startup ordering

  • Persistent volumes — models, custom nodes, outputs, inputs, user settings, and workflows are all mounted from the host

Would love to know if you find this useful, and also very interested in case you find additional potential optimizations that might speed things up.

2

Nice job – I’ll check it out!

3

hi thanks so much! it really helps, im not running with docker but this also works for my direct pull and install with venv

4

I’ve just noticed an error in the Docker compose. CUDA is not a valid value for CUDA_MODULE_LOADING.

5

You are correct, that was my bad. With this (wrong) setting it is defaulting to LAZY most likely, which in my testing performs better than EAGER. I will update the repo, sorry for that.

6

No need to be sorry, I’m sure everybody reading this thread appreciates the effort you put into this :)

7

Hi - thanks for this!

I found that while running this with 50G memory at the same time as vLLM (at 0.3 gpu memory) that the Flux model partially offloaded for each image generation - despite seemingly having enough RAM. Claude Code helped me fix it, so wanted to share the fix with you (verified through testing). See below.

Symptom

With ComfyUI alone, sequential jobs at 1080x1350, 5 steps, regional Flux2 workflow, varying prompts across 3-6 jobs: models stay resident, ~19-25s per image, log reports loaded completely; ... full load: True.

With vLLM co-resident, every ComfyUI job shows the text encoder partially offloaded, sampling time roughly doubles, and “Unloaded partially” messages appear between jobs:

Requested to load Flux2TEModel_
loaded partially; 4072 MB usable, 4072 MB loaded, 3600 MB offloaded, 237 MB buffer reserved

Per-image time grows from ~20s clean to 100-330s.

Root cause

comfy/model_management.py::get_free_memory() uses torch.cuda.mem_get_info() (wrapping cudaMemGetInfo) to decide whether to keep a model resident on “GPU” vs offload it.

On unified-memory systems, cudaMemGetInfo reports only the memory not currently allocated by any CUDA process on the same device. When vLLM has 34 GB bound, cudaMemGetInfo returns roughly 6 GB free - even though the host’s unified memory pool still has 40+ GB of real headroom.

ComfyUI then concludes the text encoder (~7.7 GB) can’t fit, offloads part of it to “CPU” (which on unified memory is the same physical RAM it just decided it couldn’t keep on GPU), and every forward pass pays the partial-offload penalty.

Confirmed directly via /system_stats:

vram_free:  6.2 GB   <- what CUDA reports
ram_free:  46.3 GB   <- what the host actually has free

Fix

Replace the CUDA memory query with psutil.virtual_memory().available in the CUDA branch of get_free_memory(). On GB10 these are semantically the same pool; psutil’s view correctly accounts for system reality including other processes’ allocations.

Patch

Here is the patch I have successfully used in the Docker container:

# ---- Patch model_management.py get_free_memory() for unified memory (GB10/DGX Spark) ----
# cudaMemGetInfo under-reports free memory on unified-memory systems when another
# CUDA process (e.g. vLLM) is resident - reported free is ~6GB when host actually
# has 40+GB available. ComfyUI then offloads text encoder partially and sampling
# slows ~2x. Replacing the CUDA call with psutil's host-RAM available gives the
# correct number on GB10 where system and GPU memory are the same pool.
RUN python - <<'PY'
from pathlib import Path

path = Path("/opt/ComfyUI/comfy/model_management.py")
text = path.read_text()
old = "mem_free_cuda, _ = torch.cuda.mem_get_info(dev)"
new = "import psutil as _psutil; mem_free_cuda = _psutil.virtual_memory().available"
if old not in text:
    raise SystemExit("Expected pattern not found in comfy/model_management.py")
path.write_text(text.replace(old, new, 1))
PY
8

Curious as to the best models to use for video generations. I’ve found lots of review but would be keen to see what folks here use.

9

Hi Luix93

Thanks a lot,
by using your well explained settings on my own docker, the stability is back again.
I’m so happy.

Regards,
Peter

10

Hi Luix93, I followed your process to generate a new ComfyUI, and it went relatively smoothly. The ComfyUI version I generated might be quite new, v0.27. Now, the problem is that version v0.27 does not support fp8 or fp4 precision, and it shows that Comfy Kitchen needs to be updated to 0.2.61. I would like to ask how Comfy Kitchen is compiled. I saw a version in the Git repository, but it doesn’t work in the new version anymore. Looking forward to your reply, thank you!

12

Hi, Thanks for the setup and it is working for me. But it is frustrating that I need to do AllUpdates everytime I do docker compose down and up again. How can we persist the updates ?