Recipe: GLM-5.2 on 4× DGX Spark at 1M Context with sparkrun (+ Vision)

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

I’ve been running GLM-5.2 at 1M context on my four-node DGX Spark cluster with Hermes for several weeks of day-to-day work. I wanted to share a streamlined, stability-tested sparkrun recipe for anyone looking to deploy the same setup with only a handful of commands:

The repository uses @ciprianveg’s Docker image and includes a recipe for the text-only QuantTrio/GLM-5.2-Int4-Int8Mix model. It also supports adding the vision tower from baseten/GLM-5.2-Vision-NVFP4; see Baseten’s GLM-5.2 vision blog post for background. Finally, it comes set up out of the box with Adaptive MTP.

This repository builds on work by @ciprianveg, @CosmicRaisins, @Zatz, and @tonyd615 – a big thank-you to all of them. Their contributions are also acknowledged in the repository.

I plan to update the repository for GLM-5.3 once a suitable quantization becomes available.

2

Could you share benchmarks using tool-eval-bench?

3

Running it now – will follow up when it’s done.

4

Here are the results from a run using the text-only model with no adjustments on the latest tool-eval-bench (run command: tool-eval-bench run --seed 42 --base-url ...).

Model: QuantTrio/GLM-5.2-Int4-Int8Mix
Score: 86 / 100
Rating: ★★★★ Good
Benchmark: tool-eval-bench v2.5.1.dev31+g02644a320
Engine: vLLM 0.11.2.dev280+gilded.gnosis.v18.vllm264bce1.b12xbc85ef3.fi801d57a.cu132.20260718
Quantization: INT8
Max context: 1,048,576 tokens

✅ 53 passed ⚠️ 12 partial ❌ 4 failed
Points: 118/138

Quality: 86/100
Responsiveness: 33/100 (median turn: 4.7s)
Deployability: 70/100 (α=0.7)
Weakest: M Autonomous Planning (50%)

Completed in 1262.9s

📊 Token Usage:
Total: 258,313 tokens
Efficiency: 0.5 pts/1K tokens

5

TTFT and tok/s code vs prose?

6

Do you have a preferred benchmark for that? Here’s a very quick run from llama-benchy (fifth run to let the Adaptive MTP settle after the tool-eval-bench):

uv run llama-benchy --base-url http://.../v1 --model QuantTrio/GLM-5.2-Int4-Int8Mix

llama-benchy (0.4.1.dev1+ge9be34457)
Date: 2026-08-17 21:10:10
Benchmarking model: QuantTrio/GLM-5.2-Int4-Int8Mix at http://.../v1
Concurrency levels: [1]
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading text from cache: /home/daved/.cache/llama-benchy/cc6a0b5782734ee3b9069aa3b64cc62c.txt
Total tokens available in text corpus: 141280
Warming up...
Warmup (User only) complete. Delta: 14 tokens (Server: 35, Local: 21)
Warmup (System+Probe) complete. Delta: 15 tokens (Server: 37, Local context: 21, Probe: 1)

Running coherence test...
Coherence test PASSED.
Measuring latency using mode: api...
Average latency (api): 7.36 ms
Running test: pp=2048, tg=32, depth=0, concurrency=1
  Warmup 1/1 (batch size 1)...
  Run 1/3 (batch size 1)...
  Run 2/3 (batch size 1)...
  Run 3/3 (batch size 1)...
Printing results in MD format:



| model                          |   test |            t/s |     peak t/s |        ttfr (ms) |     est_ppt (ms) |    e2e_ttft (ms) |
|:-------------------------------|-------:|---------------:|-------------:|-----------------:|-----------------:|-----------------:|
| QuantTrio/GLM-5.2-Int4-Int8Mix | pp2048 | 556.19 ± 27.75 |              | 3699.11 ± 190.90 | 3691.75 ± 190.90 | 3699.11 ± 190.90 |
| QuantTrio/GLM-5.2-Int4-Int8Mix |   tg32 |   22.17 ± 2.35 | 24.67 ± 2.05 |                  |                  |                  |

llama-benchy (0.4.1.dev1+ge9be34457)
date: 2026-08-17 21:10:10 | latency mode: api
7

Single run AIME25 benchmark:

sgl-eval run aime25 \
  --model QuantTrio/GLM-5.2-Int4-Int8Mix \
  --n-repeats 1 \
  --num-threads 5 \
  --max-tokens 64000 \
  --temperature 1.0 --top-p 0.95 --thinking \
  --out-dir /home/daved/logs \
  --base-url http://.../v1

Run directory: /home/daved/logs/sgl_eval_aime25_20260818-035247
aime25: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [1:24:15<00:00, 168.50s/it, acc=90.00%]
== aime25 ==
30 examples (single-shot)  |  5055.1s  |  57 tok/s  |  290K tokens

* score           =  90.00%
  stop_rate       =  100.00%
  truncated_rate  =  0.00%
  error_rate      =  0.00%

Metrics: /home/daved/logs/sgl_eval_aime25_20260818-035247/metrics.json
Predictions: /home/daved/logs/sgl_eval_aime25_20260818-035247  (1 jsonl file(s))
8

Thanks for the attribution. Mind linking my name to my GitHub profile instead of my forum account?

9

Definitely – I was actually wondering which one people would prefer. All done on the repo!

Thanks again for everything you do for the community!

10

thank you for this