A new Chisel recipe serves Qwen3.8-27B on a single DGX Spark. On spark-arena it is now the fastest single-Spark entry for this model: https://spark-arena.com/benchmark/sub1792119669798
spark-arena numbers (llama-benchy, 128 output tokens)
- Ranking test (16k context, 2 users): 60.6 tok/s total
- 1 user, no context: 42.2 tok/s (23.7 ms/token), prefill 2,158 tok/s
- 1 user at 16k: 33.4 tok/s; at 100k: 29.5 tok/s
For comparison, my earlier FP8 recipe (topic) scored 28.3 on the same ranking test.
Quality against BF16 (lm-evaluation-harness, chat template, thinking off, full tasks)
| Task | Recovery vs BF16 |
|---|---|
| gsm8k (5-shot) | 99.3% |
| ifeval (prompt-level strict) | 101.1% |
| humaneval (chat) | 99.4% |
| LongBench NarrativeQA (full stories, F1) | 101.2% |
None is clearly worse than BF16; per-question answers are in the repo.
What’s in it
- Weights: unsloth/Qwen3.8-27B-NVFP4.
- DFlash2 speculative decoding with 10 drafts, using Chisel’s NVFP4 build of z-lab’s drafter (1.4 GB instead of 3.8 GB): yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter · Hugging Face
- Linear-attention state cached in BF16 (
--mamba-ssm-cache-dtype bfloat16). - A small Chisel vLLM plugin on the spark-arena nightly image: it rebuilds the linear-attention state only for accepted drafts instead of storing it per draft token, and gives the drafter a 100k-token FP8 output head. Both leave the target’s output unchanged.
- Builds on Luis Poveda’s single-Spark recipe. Thanks to unsloth, z-lab and the vLLM team.
On my earlier DFlash2 long-context question: on the nightly image the NarrativeQA answers are clean with prefix caching on, unlike what I saw on vLLM 0.31.0 (vllm#60174).
Run it
sparkrun registry add https://github.com/yuanlin2004/chisel-spark-recipes
sparkrun run @chisel/chisel-nvfp4-dflash2-replay --tp 1 --hosts localhost
The first run downloads unsloth’s 22 GB checkpoint.
Questions and results from other Sparks welcome.