GLM-5.3 (743B) Int4-Int8Mix on 4x DGX Spark (GB10) TP4, up to 46 tok/s @ 200K ctx

Following the GLM-5.2 GB10 recipe thread, here’s the full GLM-5.3 (the big 743B flagship, not Flash) quantized and serving on four DGX Spark (GB10 / sm121 / aarch64). As far as I can tell it’s the first quant of the large 5.3 that actually serves on 4x GB10 with real KV headroom. Weights and the full recipe are both public.

**What it is**

  • Base: zai-org/GLM-5.3-BF16 (genuine BF16, 1507 GB). GlmMoeDsaForCausalLM, 78 layers + MTP, 256 routed + 1 shared experts, hidden 6144, kv_lora_rank 512.
  • Quant: Int4-Int8Mix, compressed-tensors / pack-quantized, data-free RTN (no calibration set), group 128, symmetric, weight-only.
  • Output: 377.4 GiB, 282 shards. ~95.5 GiB/rank at TP4, leaving ~33 GiB/rank for KV.

**Recipe (config_groups taken verbatim from QuantTrio/GLM-5.2-Int4-Int8Mix)**

  • W4A16 on the MoE experts (layers 3-77), the bulk of the weights.
  • W8A16 on attention + dense + shared-expert MLP (layers 1-77).
  • W8A16 channelwise on the MTP block (layer 78).
  • FULL precision (what protects accuracy with no calibration): layer 0, every mlp.gate (router), self_attn.indexer + indexers_proj (DSA selector), MTP norms, lm_head.
  • kv_cache_scheme None (KV precision is a serve-time choice).
  • Produced with a shard-streaming RTN quantizer (not oneshot, which would need ~1.4 TB offload scratch): read one BF16 shard, quantize with compressed-tensors’ own calculate_qparams/quantize/pack_to_int32, write a 1:1 shard. ~10 GiB peak RAM, 28.2 min.

**Serving (vLLM, TP4, no Ray)**
–quantization compressed-tensors --kv-cache-dtype fp8_ds_mla --tensor-parallel-size 4, MTP k=4, cudagraph FULL, 200K ctx. On sm121 you need the sm12x sparse-MLA kernel overlays (stock vLLM faults) from the GLM-5.2 repo below; this recipe changes only the weights path.

**Benchmark of record (thinking off, all ranks clocking correctly)**
c1 12.12 / c2 21.71 / c3 28.30 / c4 33.11 / c6 46.03 tok/s aggregate, 200K ctx, fp8_ds_mla KV. GLM-5.2 QuantTrio on the same 4 nodes: 32.5 mean / 36 peak.

**GB10 gotchas (each cost real time)**

  • Unified memory: host page cache eats CUDA-visible memory 1:1. Run an unconditional drop_caches flusher every 60s or you silently lose gmu headroom.
  • Pin --kv-cache-memory-bytes for deterministic boots.
  • vm.swappiness=10 (default 0-1 wedges with swap untouched).
  • A degraded GPU clock after a crash (one rank stuck ~720 MHz vs 2400+) is reboot-only, and in TP4 drags the whole cluster ~25%. Check per-node clocks.sm under load.

**Honest caveats**
Data-free RTN, no calibration. Coherence proven; formal eval vs the fp8 original has NOT been run, so no parity claim. DFlash2 spec-decode is one flag away (fp8_e4m3 KV, method “dflash”, num_speculative_tokens 7) and is next.

**Links**

**Credit**
Recipe map from QuantTrio. Base zai-org. compressed-tensors by Neural Magic. sm12x sparse-MLA kernels by CosmicRaisins (via the GLM-5.2 GB10 repo) + the thread 374125 contributors (ciprianveg, eugr, and others). DFlash2 drafter by IncoAI.

great! I was downloading the same model! thanks for your service tony!