No DFlash2 post exists yet for this model on GB300-class hardware. The published recipes (tonyd2wild on vLLM (h/t @tonyd615), beastllama on SGLang) target DGX Spark / GB10 at SM121. This is the SM100 lane: GLM-5.3-Flash NVFP4 with an incoai DFlash2 drafter on a single GB300, TP1.
The bit that decides everything on the Station: the MXFP8 drafter needs the B12X kernel stack, which is SM120/SM121-only and will not run on GB300 (SM100). So on the Station you use the BF16 incoai drafter. That is what this is.
Stack
- Target:
local-inference-lab/GLM-5.3-Flash-NVFP4(calibrated, ~169B), fp8 KV, 1M context - Drafter:
incoai/GLM-5.3-Flash-DFlash2(BF16 block-diffusion, 7-token blocks) - Runtime: vLLM 0.1.dev20051+g487ecf187 (the glm53-flash image) plus a ported DFlash2 overlay, tonyd2wild’s GB10 overlay adapted to SM100 geometry
- Build: ~25s overlay on the public
vllm/vllm-openai:glm53-flashbase. No source build, no B12X.
Numbers (single GB300, TP1, temp 0, stream:false, no-think)
| config | decode |
|---|---|
| no spec | ~137 tok/s |
| DFlash2 (7-token blocks) | ~305-470 tok/s |
Draft acceptance runs 54-67%, rising warm, with healthy per-position decay across the 7-token block (207 down to 96). The drafter, target, and rejection sampler are wired correctly on SM100. The range reflects prompt, context, and RTT; code-style prompts land at the top.
The port’s hard part is done up front. GLM-5.3’s custom KV grouping (34 KDA/mamba + 11 MLA slot-sharing) is the DFlash2 minefield. I re-derived the exact-fit geometry for GB300 and made it a build-time check (sim_glm5_drafter_hades.py): drafter block 2048 to 4288, real page == mla_page (4,390,912 B), per-block KV cost unchanged. The build fails if it ever breaks. First boot came up clean.
Everything (overlay patches, GB300 geometry sim, stack recipe) is in a prebuilt image: ghcr.io/ebfio/glm53-flash-dflash2-gb300:latest (pin sm100-gb300-20260901).
Compose change vs your non-spec glm53-flash compose: swap the image and add
--speculative-config '{"method":"dflash","model":"/models/.../incoai--GLM-5.3-Flash-DFlash2/snapshots/<snap>","num_speculative_tokens":7}'
num_speculative_tokens must be 7 (drafter block 8 minus the target’s own token).
Caveats
- DFlash2 drafts text only. Image and video requests pass through unspeculated.
- Long-prefill stability past ~32K tokens on this stack is fully verified!
incoai/GLM-5.3-Flash-DFlash2is CC BY-NC-ND 4.0 (non-commercial). Respect the terms.- Validated on one GB300 at TP1. Other SM100 configs are untested.
- Credit to tonyd2wild’s GB10 overlay this is ported from, and to incoai for DFlash2. Happy to share more numbers or logs for anyone reproducing on a Station.