Following the GLM-5.2 GB10 recipe thread, here’s the full GLM-5.3 (the big 743B flagship, not Flash) quantized and serving on four DGX Spark (GB10 / sm121 / aarch64). As far as I can tell it’s the first quant of the large 5.3 that actually serves on 4x GB10 with real KV headroom. Weights and the full recipe are both public.
**What it is**
- Base: zai-org/GLM-5.3-BF16 (genuine BF16, 1507 GB). GlmMoeDsaForCausalLM, 78 layers + MTP, 256 routed + 1 shared experts, hidden 6144, kv_lora_rank 512.
- Quant: Int4-Int8Mix, compressed-tensors / pack-quantized, data-free RTN (no calibration set), group 128, symmetric, weight-only.
- Output: 377.4 GiB, 282 shards. ~95.5 GiB/rank at TP4, leaving ~33 GiB/rank for KV.
**Recipe (config_groups taken verbatim from QuantTrio/GLM-5.2-Int4-Int8Mix)**
- W4A16 on the MoE experts (layers 3-77), the bulk of the weights.
- W8A16 on attention + dense + shared-expert MLP (layers 1-77).
- W8A16 channelwise on the MTP block (layer 78).
- FULL precision (what protects accuracy with no calibration): layer 0, every mlp.gate (router), self_attn.indexer + indexers_proj (DSA selector), MTP norms, lm_head.
- kv_cache_scheme None (KV precision is a serve-time choice).
- Produced with a shard-streaming RTN quantizer (not oneshot, which would need ~1.4 TB offload scratch): read one BF16 shard, quantize with compressed-tensors’ own calculate_qparams/quantize/pack_to_int32, write a 1:1 shard. ~10 GiB peak RAM, 28.2 min.
**Serving (vLLM, TP4, no Ray)**
–quantization compressed-tensors --kv-cache-dtype fp8_ds_mla --tensor-parallel-size 4, MTP k=4, cudagraph FULL, 200K ctx. On sm121 you need the sm12x sparse-MLA kernel overlays (stock vLLM faults) from the GLM-5.2 repo below; this recipe changes only the weights path.
**Benchmark of record (thinking off, all ranks clocking correctly)**
c1 12.12 / c2 21.71 / c3 28.30 / c4 33.11 / c6 46.03 tok/s aggregate, 200K ctx, fp8_ds_mla KV. GLM-5.2 QuantTrio on the same 4 nodes: 32.5 mean / 36 peak.
**GB10 gotchas (each cost real time)**
- Unified memory: host page cache eats CUDA-visible memory 1:1. Run an unconditional drop_caches flusher every 60s or you silently lose gmu headroom.
- Pin --kv-cache-memory-bytes for deterministic boots.
- vm.swappiness=10 (default 0-1 wedges with swap untouched).
- A degraded GPU clock after a crash (one rank stuck ~720 MHz vs 2400+) is reboot-only, and in TP4 drags the whole cluster ~25%. Check per-node clocks.sm under load.
**Honest caveats**
Data-free RTN, no calibration. Coherence proven; formal eval vs the fp8 original has NOT been run, so no parity claim. DFlash2 spec-decode is one flag away (fp8_e4m3 KV, method “dflash”, num_speculative_tokens 7) and is next.
**Links**
- Weights: 2wild4tv/GLM-5.3-Int4-Int8Mix · Hugging Face
- Recipe + launchers + benchmarks: GitHub - tonyd2wild/GLM-5.3-Int4-Int8Mix-TP4-4x-DGX-Spark: First Int4-Int8Mix quantization of the full GLM-5.3 (743B), served on 4x NVIDIA DGX Spark (GB10) at TP4. Quantizer, verification gates, and serving recipe. · GitHub
**Credit**
Recipe map from QuantTrio. Base zai-org. compressed-tensors by Neural Magic. sm12x sparse-MLA kernels by CosmicRaisins (via the GLM-5.2 GB10 repo) + the thread 374125 contributors (ciprianveg, eugr, and others). DFlash2 drafter by IncoAI.