A dense 27B reads all 27 billion parameters per token. That is why it is good, and why it is unusable out of the box on one Spark. This is a measured configuration that fixes it, installed by one script — v1.0: albond/DenseSpark-Qwen3.8-27B · GitHub
Why this model. On Artificial Analysis’s Intelligence Index v4.1.1: Qwen3.8-27B scores 52 at high reasoning effort, Qwen3.5 122B A10B scores 33, Flash-Next scores 56 at 180B resident. 27B dense is the size one Spark holds whole, four points off a model seven times larger. Those points cost reasoning tokens — and length is only affordable if tokens arrive fast.
16 concurrent requests. NVIDIA’s three shapes, output tok/s. Stock figures as published on this forum; mine are the median of three retained runs after a warm-up, on the image the installer builds.
| test | stock FP8 | stock NVFP4 | this | vs NVFP4 |
|---|---|---|---|---|
| balanced (1k/1k) | 104.44 | 134.41 | 260.2 | 1.94x |
| decode-heavy (1k/8k) | 99.47 | 132.07 | 227.0 | 1.72x |
| prompt-heavy (8k/1k) | 65.58 | 87.91 | 111.2 | 1.26x |
Single request — 58.1 tok/s across five prompts, three runs each, decode alone. Full spread, not the best of it:
| prompt | tok/s | TTFT |
|---|---|---|
| Q&A 256 | 36.6 | 217 ms |
| LongCode 2048 | 49.6 | 249 ms |
| Code 512 | 51.0 | 248 ms |
| Math 64 | 75.5 | 243 ms |
| JSON 1024 | 77.9 | 249 ms |
Token-weighted 54.8. First token in ~0.25 s; a 2,000-token answer in ~35 s — about ten times faster than you read.
Where stock wins: time to first token, all three shapes. NVFP4 4.05 s vs my 7.57 (balanced), 29.81 vs 37.97 (prompt-heavy), 4.05 vs 7.56 (decode-heavy). The prefill path that would have closed it was built, measured, dropped — it fails the quality gate. If your load is many short requests, weigh that first.
Quality.
- Runtime: paired mean-NLL delta −0.00012 nat/token against exact dequantization of its checkpoint, threshold 0.005, over 23,997 tokens. No measurable degradation on that gate — not proof every answer is identical, not a task benchmark.
- Checkpoint: NVFP4 and FP8 are formats — a fixed rule converts the tensor. AutoRound is a calibrated method: it searches rounding decisions against calibration data, so quality is a property of how the build was made, not only of its bit count. A stock 4-bit build is already a good starting point.
- You are not stuck with mine. Quantize your own with your own calibration set and point the installer at it. The kernels take any checkpoint meeting their contract — symmetric GPTQ, 4-bit, group 128, BF16 activations, no act-order, no zero points, this model’s geometry. That knob does not exist with a fixed-format checkpoint.
- I have not measured this checkpoint against the FP8 or NVFP4 builds on quality, so no ranking between them is claimed here.
How. AutoRound INT4 weights; the checkpoint’s built-in MTP layer proposing tokens the full model verifies; SM121-specific kernels for the output projection, the linear attention 48 of 64 layers use, and the backbone matmuls, each dispatched by row count.
Use.
git clone https://github.com/albond/DenseSpark-Qwen3.8-27B
cd DenseSpark-Qwen3.8-27B
./install.sh
It asks one question — how many concurrent requests you expect — then ./configs/launch-densespark.sh serves an OpenAI-compatible API. That question is not cosmetic: speculation depth is selected per concurrency, 8 at one request and 3 from three up, because a long proposal pays off when the machine would otherwise wait and wastes work when sixteen keep it busy.
Not claimed. Concurrency above 16, context beyond ~9k tokens, multimodal, task accuracy, any other machine. Negative results are in the repo with reasons — including the faster prefill path that is not shipped, and a draft-head variant withdrawn for killing the engine on deterministic requests.