GLM 5.2 with vision!

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Saw this gem on hf baseten/GLM-5.2-Vision-NVFP4 · Hugging Face.

This checkpoint adds it, without touching a single GLM weight: the text backbone and the vision tower are both frozen and byte-identical to their upstream releases. The only newly-trained parameters are the 49.5M-parameter projector that maps MoonViT’s 1152-dim patch embeddings into GLM’s 6144-dim token space.

I ported over their change to my repo for a 4-node cluster: GitHub - CosmicRaisins/glm-5.2-gb10: GLM-5.2 (744B/40B MoE) on a 4× DGX Spark / GB10 (sm_121) cluster: portable Triton sparse-MLA kernels, a data-free expert prune, MTP draft, and a one-script bootstrap. · GitHub

I also modified @aidendle94’s work on adaptive mtp. The model will dynamically switch between 2 and 5 drafted tokens depending on the acceptance of p2 to p4. Now you can get the 30+ tok/s speed up in code without giving up performance in prose, theoretically.

I’ve only been using this stack for a day, and only simple OCR and classification testing with the vision encoder, so give it a try and let me know how it works!

2

Awesome! I am eager to see if the vision is good enough for gui screenshot feature implementations and fixing visual bugs without the need to have other model describing the image. Also making the mtp adapter to switch should bring a speedup in coding. Thank you!

3

@CosmicRaisins, @ciprianveg Have either of you continued using the vision tower with GLM 5.2 for actual development tasks? I’m considering testing them for real-world tasks, but the scuttlebutt on the DS4 Flash vision variants on the forum suggested these may not be production-ready for agentic workflows (e.g., browser automation with vision-based click coordinates).

4

I used it and it behave correctly. In opencode.