Hey guys. I was recently inspired by the NVFP4/FP3 mixed precision model worked on by the guys working on B12X to get GLM 5.2 to run on 4x RTX6Ks. For DGX Sparks we have some more headroom so I made a mixed precision model. This allows for more weight savings without compromising too much on precis…

Structured Output 58% — it’s the reasoning parser, not the model @alexander.korolev.germany — I ran the same benchmark on the same model and engine and got 58% on Structured Output too. Exactly 58%, same as you. Different hardware, same number. That bugged me enough to stop looking at the score and …

thank you for this. giving it a run today

Great job, @aidendle94 ! Getting useful GLM 5.2 would be really beneficial for 4xSpark setups. Also in the process of getting it to work. Why docker lacks unset functionality? It is a real problem and there is no normal solution, really frustrating.

Kinda, I created a Dockerfile which adds another entrypoint: FROM aidendle94/sparkrun-vllm-ds4-gb10:production-hybrid-1.2 # Wrapper: unset the baked var, then hand off to the original entrypoint. # The original nvidia_entrypoint.sh does `exec "$@"` at the end, so this # runs as PID 1 with conda/CU…

I’m getting ~800 prefill but ~20 decode (sad) in llama-benchy. What’s your experience with NVFP4 KV? Any issues below 262k ctx?

Are you using FP8 kv? I haven’t had any issues with NVFP4 KV. I implemented some work on top to make sure that it’s scaled and calibrated correctly based on the work by Koush and Dooner and based on their test results its almost noise.

I’m using fp8 with my own image, but launched with nvfp4 with yours. I think it’s probably a prose difference. I wasn’t getting very high acceptance with llama-benchy’s default prose. I just noticed that you have adaptive speculative depth, that’s huge!!!

Have a v2 model coming out with lower KLD…trying to bring it down as much as possible.

Have you run glm 5.2 on a 6 node setup? Know what the t/s would be for concurrent? I am trying to figure out if another two sparks would get me several streams of 20 t/s

GLM 5.2 Hybrid-FP8+NVFP4+MXFP4 + Optimal runtime recipe

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

aidendle94 July 21, 2026, 8:15pm 4

Sorry, were you able to resolve this? What issues did you run into?

Topic		Replies	Views
GLM-5.2 on a 3× GB10 cluster: ~16,13 tok/s decode, 215K ctx FP8 TP=3 + VISION! DGX Spark / GB10 Projects	7	1274	July 26, 2026
GLM-5.2 on a 4× GB10 cluster: ~22 tok/s decode, 256K ctx, Recipe DGX Spark / GB10 llama , deepseek	200	13279	August 9, 2026
GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode (generic - coding/structured) DGX Spark / GB10 gaming , llama	29	2270	August 29, 2026
(Academic) GLM-5.2 on 2x DGX-Spark/GB10 nodes: Crazy 1-bit UD-IQ1_S + RPC llama.cpp + 256K context + 8 tok/s DGX Spark / GB10 inception , llama , deepseek	8	3925	August 1, 2026
GLM-5.2 (unpruned) @ 200K context on 4× DGX Spark — 27 tok/s single / 52.5 tok/s @c4 DGX Spark / GB10 Projects deepseek	3	748	August 4, 2026
GLM-5.3-Flash: 320B total parameters / 18B active DGX Spark / GB10	286	11815	September 5, 2026
Fitting a high-quality REAP-less GLM-5.2 onto 4x DGX Spark DGX Spark / GB10 deepseek	0	2084	June 29, 2026
Running GLM-4.7-FP8 (355B MoE) on 4x DGX Spark with SGLang + EAGLE Speculative Decoding DGX Spark / GB10 Projects	38	2925	June 24, 2026
GLM-5.3-Flash weights released (Ox Alpha) DGX Spark / GB10 Projects	10	4848	August 28, 2026
How to run GLM 4.7 on dual DGX Sparks with vLLM / mods support in spark-vllm-docker DGX Spark / GB10	27	4813	January 2, 2026

GLM 5.2 Hybrid-FP8+NVFP4+MXFP4 + Optimal runtime recipe

Related topics