Z-Image Turbo NVFP4

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
,
1

I wanted to run Z-Image Turbo on my spark and keep it resident in memory for image gen on call. bf16 meant ~23 GB peak which was too big. I found an NVFP4 from ComfyUI, but didn’t want to deal with ComfyUI here for a simple service. Here’s a recipe to get it down to ~10 GB peak using NVFP4 from ComfyUI, but without using ComfyUI.

Z-Image Turbo in NVFP4 on DGX Spark (GB10), no ComfyUI

Z-Image Turbo through plain diffusers + comfy-kitchen kernels, quantized to NVFP4. ~7.5 GB resident, ~19 s for a 1024² 8-step image, quality intact. No ComfyUI, no nunchaku.

Environment. GB10 is SM_121 / CUDA 13.0 / aarch64. Stock PyTorch cu130 wheels just work… no source build:

uv venv .venv --python 3.12 && . .venv/bin/activate

uv pip install torch==2.11.0 torchvision --index-url https://download.pytorch.org/whl/cu130

python -c "import torch; print(torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))"

# NVIDIA GB10 (12, 1)

diffusers has native Z-Image support, but only on git main:

uv pip install "diffusers @ git+https://github.com/huggingface/diffusers" transformers accelerate safetensors

bf16 baseline (works immediately).

from diffusers import ZImagePipeline; import torch

pipe = ZImagePipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", torch_dtype=torch.bfloat16).to("cuda")

img = pipe("a grey heron at dusk", num_inference_steps=8, guidance_scale=1.0, height=1024, width=1024).images[0]

~23 GB peak. Turbo wants 8 steps / guidance ~1.0 — the pipeline defaults (50 / 5.0) are for the base model.

Trap: the GGUFs don’t load. `leejet/Z-Image-Turbo-GGUF` & co. are the *original* Z-Image layout, dim 2160, fused qkv. diffusers’ `ZImageTransformer2DModel` is dim 3840 with split `to_q/k/v`. `from_single_file` maps the names but the shapes mismatch and it won’t load. Don’t bother. Quantize the native diffusers weights yourself.

NVFP4 via comfy-kitchen. Its `scaled_mm_nvfp4` routes through cuBLASLt, which works on GB10. (CUTLASS FP4 is gated to `sm_100a` and does *not* — but you never hit that path.) Build from source; two non-obvious requirements:

sudo apt install -y python3.12-dev        # nanobind/CMake needs Python.h

uv pip install "setuptools>=61" wheel "nanobind>=2.0.0" cmake ninja   # venv cmake 4.x; system 3.28 is too old for 120f

git clone  && cd comfy-kitchen

PATH="$VIRTUAL_ENV/bin:$PATH" uv pip install -e ".[cublas]" --no-build-isolation

Arch gotcha: it builds via CMake `–cuda-archs`, default `…;120f` — that family target covers SM_121. It ignores `TORCH_CUDA_ARCH_LIST`. Check:

import comfy_kitchen as ck

print(ck.list_backends()["cuda"])   # available=True, 'scaled_mm_nvfp4' in capabilities

Quantize and run a linear:

from comfy_kitchen.tensor import QuantizedTensor

qw = QuantizedTensor.from_float(linear.weight.data, "TensorCoreNVFP4Layout")  # note: STRING, not the class

# F.linear(x_q, qw) dispatches to scaled_mm_nvfp4 — the activation must ALSO be a 2D NVFP4 QuantizedTensor

Wrap each `layers.*` Linear (both the DiT *and* the Qwen3-4B text encoder) in a module that quantizes the reshaped-2D activation per call, then `F.linear(xq, qw, bias)`. Leave norms / embeddings / `context_refiner` bf16. cos vs bf16 ≈ 0.99; images hold.

Comfy-Org also ships a pre-quantized `z_image_turbo_nvfp4.safetensors` with *calibrated* static activation scales, but it’s in that same original fused-qkv layout — and dynamic activation quant (scale computed per call from the activation’s own range) landed at ~0.99 cos and held up visually, so the calibration wasn’t worth the layout-splitting work; skip it unless you measure a quality gap.

Don’t offload; do save the checkpoint:

  • cpu offload is useless here — moving the 8 GB text encoder GPU<->CPU measured ~50 s *each way*. Unified memory, but the migration is brutal. Keep everything resident.
  • Quantize once, save, lean-load. Self-quantizing at load spikes to full bf16. Instead save the NVFP4 tensors (qdata + block_scale + per-tensor scale, via the layout’s `state_dict_tensors`), rebuild the model on `meta`, replace the quantized linears, and `load_state_dict(…, assign=True)` the bf16 remainder — bf16 never materializes. Catch: transformers’ Qwen3 rotary `inv_freq` is a non-persistent buffer, so it stays on `meta` after assign — rebuild it via `Qwen3RotaryEmbedding(config, device=“cuda”)`.

Numbers (full NVFP4, DiT + text encoder):

  • idle resident: 7.5 GB
  • peak (generating): 10.1 GB
  • 1024² / 8 steps: ~19 s
  • cold load: ~47 s

Versions: torch 2.11.0+cu130, diffusers 0.39.0.dev0 (git), transformers 5.9.0, comfy-kitchen 0.2.10, nvcc 13.0.88 as of May 30, 2026

2

Thanks.