Qwen3.8-27B at 60.6 tok/s on one DGX Spark (fastest single-Spark entry on spark-arena), quality checked against BF16

A new Chisel recipe serves Qwen3.8-27B on a single DGX Spark. On spark-arena it is now the fastest single-Spark entry for this model: https://spark-arena.com/benchmark/sub1792119669798

spark-arena numbers (llama-benchy, 128 output tokens)

  • Ranking test (16k context, 2 users): 60.6 tok/s total
  • 1 user, no context: 42.2 tok/s (23.7 ms/token), prefill 2,158 tok/s
  • 1 user at 16k: 33.4 tok/s; at 100k: 29.5 tok/s

For comparison, my earlier FP8 recipe (topic) scored 28.3 on the same ranking test.

Quality against BF16 (lm-evaluation-harness, chat template, thinking off, full tasks)

Task Recovery vs BF16
gsm8k (5-shot) 99.3%
ifeval (prompt-level strict) 101.1%
humaneval (chat) 99.4%
LongBench NarrativeQA (full stories, F1) 101.2%

None is clearly worse than BF16; per-question answers are in the repo.

What’s in it

  • Weights: unsloth/Qwen3.8-27B-NVFP4.
  • DFlash2 speculative decoding with 10 drafts, using Chisel’s NVFP4 build of z-lab’s drafter (1.4 GB instead of 3.8 GB): yuanlin2004/Qwen3.8-27B-Chisel-DFlash2-NVFP4-drafter · Hugging Face
  • Linear-attention state cached in BF16 (--mamba-ssm-cache-dtype bfloat16).
  • A small Chisel vLLM plugin on the spark-arena nightly image: it rebuilds the linear-attention state only for accepted drafts instead of storing it per draft token, and gives the drafter a 100k-token FP8 output head. Both leave the target’s output unchanged.
  • Builds on Luis Poveda’s single-Spark recipe. Thanks to unsloth, z-lab and the vLLM team.

On my earlier DFlash2 long-context question: on the nightly image the NarrativeQA answers are clean with prefix caching on, unlike what I saw on vLLM 0.31.0 (vllm#60174).

Run it

sparkrun registry add https://github.com/yuanlin2004/chisel-spark-recipes

sparkrun run @chisel/chisel-nvfp4-dflash2-replay --tp 1 --hosts localhost

The first run downloads unsloth’s 22 GB checkpoint.

Recipe page: chisel-spark-recipes/recipes/qwen3.8-27b/chisel-nvfp4-dflash2-replay.md at main · yuanlin2004/chisel-spark-recipes · GitHub

Questions and results from other Sparks welcome.

1 Like