Full GLM-5.3 (753B) on 4x DGX Spark - 3-bit EXL3 quant + TensorFold TP4 recipe

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

got the full GLM-5.3 running across 4 sparks and figured i’d share the formula in case anyone else wants to try it.

quant (exllamav3 1.5.3, from the zai-org BF16):
convert.py -b 3.0 -hq -hb 6 -mb 4 -cb mul1

  • routed experts: flat 3.0 bpw
  • attention / shared experts / first 3 dense layers: bumped to ~5 (-hq)
  • MTP layer 4, lm_head 6
  • ~3.04 bpw avg, 292 GB, ~70 GiB per rank at TP4

quality vs BF16 (wikitext-2, 32x2048): KLD 0.109, top-1 agreement 90.2%

serving is on a TensorFold fork (TP4 over the CX7s, MTP-3 drafting, Q5 KV cache, 1M context). single stream i’m seeing ~43 tok/s code / ~28 prose on RigMark. multi-stream is still being worked on.

weights: bertholomus/GLM-5.3-EXL3-3.0bpw · Hugging Face
recipe + scripts: GitHub - bertholomus/glm-5.3-tensorfold-tp4-4xgb10: BertholomusAI recipe: full GLM-5.3 (non-Flash) EXL3 3.0bpw on 4x NVIDIA DGX Spark (GB10), TensorFold TP4 - quant formula, launch scripts, benchmarks, measured numbers · GitHub

big credit to ashhart for TensorFold, turboderp for exllamav3 and z.ai for the model. happy to answer questions

2

That’s a really useful setup for anyone trying to run the full model across multiple DGX Sparks.
The 3-bit quantization with TP4 is a nice way to fit the model within the available memory.
43 tok/s for code on a single stream is pretty solid.
I’d be interested to see how the multi-stream results turn out once they’re optimized.