GLM-5.3-Flash (320B/18B-active, released yesterday) is serving on a pair of DGX Sparks — as far as I can find, the first published deployment of this model on GB10. The vendor recipe verifies H100/B200/GB200 only, and the arch (glm5_next: NoPE sparse-MLA + KDA linear attention) fails every attention backend the sm_121 selector offers, out of the box.
Full recipe, the vLLM mod, and honest measured numbers here:
Measured (median of 3, temp 0, single stream): 24.74 code / 30.30 structured / 19.58 prose tok/s with the checkpoint’s own MTP head at num_speculative_tokens=5 (acceptance length ~2.55). No-spec baseline is 14.6 flat — decode is MoE-bandwidth-bound on this box, CUDA graphs bought 2%, the drafter bought 1.3-2x. Needle recall exact at 8,443 tokens deep. Stock DGX Spark OS, stock per-model vLLM image (glm53-flash-arm64-cu130), no kernel builds.
Two findings that generalize beyond this model:
- SILENT FP4 MoE corruption on sm_121. The auto-selected FLASHINFER_CUTLASS NvFp4 MoE backend produces degenerate output (repeated-token loops from the first generated token) with zero errors — model loads, /health green, every completion garbage. This is the quiet sibling of the documented cudaErrorNoKernelImageForDevice gap. --moe-backend marlin fixes it. If your GB10 NVFP4 deployment “works” but outputs loops, check which MoE backend engaged before blaming your quant.
- A 60+ minute “No available shared memory broadcast block found” loop during engine boot is not necessarily a hang. Check the worker’s CPU first: at 150%+ it’s FlashInfer autotuning a kernel flavor with no cached tune (the tune persists to ~/.cache/vllm, so it pays once). We killed two perfectly healthy boots before figuring this out.
The core of the recipe is a small 4-part Python mod that dresses GLM’s NoPE-MLA (kv_lora 512 + rope 0) in DeepSeek’s fp8_ds_mla layout (512 + 64): construct the attention layer with rope=64, zero-pad q/k_pe at the handoff (zeros are rotation-invariant and add exactly 0 to every logit), and compact the kpool indexer’s 2176-wide top-k table to the compiled kernel’s 2048 (keeping the always-select-tail entries, which by construction never duplicate the main selection). Everything after that is the tested DeepSeek path.
Known limits, honestly: 32K ctx cap at publish time (probing 256K as I write this — the topk.cu shared-memory sizing issue on GB10 at long context is documented in the vllm#45317 thread), text-only (–language-model-only — the multimodal processor balloons the API front-end to 15.7 GB anon and the kernel OOM-kills it), single stream. When TRITON_MLA_SPARSE for SM12x (vllm#38476) or glm5_next mainlining (vllm#53906) lands, the mod retires.
Credits in the repo: LibertAIDAI’s NVFP4 quant (whose model card’s GB10 note was half the answer), the eugr/a3refaat spark-vllm-docker tooling, and the GLM-5.2 GB10 sparse-MLA work from CosmicRaisins/0xdfi that mapped this terrain.