Squeezing 5%~70% tok/s over SOTA by building from scratch (DS4.1Flash native precision on 4xGB10 switchless)

I asked Claude to build an inference engine from scratch (as in not using vLLM or SGLang etc.), specifically for one purpose: running DeepSeek 4.1 Flash on my exact set up (4 GB10s in a switchless ring).

Turns out this squeezes a further 5% - 70% tok/s depending on what it does, at native precision (and no further rounding beyond reference). It produces bit-exact outputs as far as I could check.

┌─────────────────────┬───────────┬───────────┬─────────────────┬──────────┐
│ row │ dsgx10 │ best │ whose │ lead │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ prose c1 │ 95.0 │ 90.2 │ knapcio v2.2 │ +5 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ code c1 │ 151.0 │ 133.2 │ knapcio v2.2 │ +13 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ structured c1 │ 185.8 │ 158.7 │ knapcio v2.4 │ +17 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ JSON c1 │ 150.2 │ 135.3 │ R5 ring (v2.3) │ +11 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ catalog, prose │ 78.1 │ 61.1 │ knapcio v2.2 │ +28 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ catalog, structural │ 122.3 │ 99.0 │ knapcio v2.2 │ +24 % │
├─────────────────────┼───────────┼───────────┼─────────────────┼──────────┤
│ prefill, real text │ 8.7k–9.5k │ 5.7k–5.9k │ spark-ds41f r6h │ 1.5–1.7× │
└─────────────────────┴───────────┴───────────┴─────────────────┴──────────┘

tok/s, one stream, greedy, sparkDash 1.8.8 prompts (prefill: 15k–120k-token real text), official checkpoint, as published by 2026-10-13. knapcio’s rows are on switched fabrics, R5 and spark-ds41f on rings, ours on a ring. Ours are the mean of two sessions; the kits’ cells are the better of two sweeps.

Provenance of every number: docs/review/external.md in the repo.

It optimises for single-stream but free and low-hanging optimisations for 2-4 streams is put in.