Stable Qwen3.8-27B NVFP4 + DFlash2 on a Single DGX Spark

Sharing my single-DGX-Spark Qwen3.8-27B configuration after more than two weeks of production use, more than 3 billion generated tokens, and sustained multi-agent concurrency.

This is the stable configuration I’ve ended up with after testing quite a few GB10 tuning combinations.

HOST

• NVIDIA DGX Spark / GB10
• Ubuntu 24.04.3
• Kernel: 6.17.0-1032-nvidia
• NVIDIA driver: 580.173.02
• CUDA 13.0
• DGX SWBUILD 7.4.0 + OTA 7.5.0

MODEL

• Target: RadixArk/Qwen3.8-27B-NVFP4
• Revision: 554ebba9b5f1b79dc11246341960360e6ef05ef4
• NVFP4 mixed precision
• Native context: 262,144
• Vision supported

DFlash2 draft:

• maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal
• Revision: bd7a934213c47a9e7ef69eef36bb3325f47fd1f1
• 16 draft tokens
• 1 speculative step
• selector_rank=256
• selector_top_k=16
• speculative_eagle_topk=1

RUNTIME

• SGLang 0.0.0.dev0+qwen38.27b.g561c8f3
• Commit c4271c3fe1262fc2adbd162c33b25de5255251c5
• DFlash2 PRs #35371 and #35496
• PyTorch 2.13.0+cu130
• Triton 3.7.1
• FlashInfer 0.6.18

SERVING CONFIGURATION

• Context: 262,144
• KV cache: fp8_e4m3
• mem-fraction-static: 0.65
• Chunked prefill: 2048
• Max prefill tokens: 16,384
• Max running requests: 8
• Attention backend: FlashInfer
• Mamba SSM: bfloat16
• Mamba radix strategy: extra_buffer
• mamba-full-memory-ratio: 11.93
• DFlash2 draft tokens: 16
• FlashInfer autotune: ON
• Sleep-on-idle: ON
• Prefill CUDA graph: OFF
• Continuous decode steps: 1
• Torch compile: OFF
• Speculative token map: not used

The resulting KV pool is approximately 722K tokens, with roughly 40 GB still available after startup and cache allocation.

CONTROLLED PERFORMANCE

On a quiet engine with the stock target:

• ~2K context: 106 tok/s
• ~65K: 92 tok/s
• ~131K: 85 tok/s
• Long free-form prose: typically 22–24 tok/s

Concurrency testing:

• 1-way: 58.5 tok/s aggregate
• 2-way: 92.3 tok/s
• 4-way: 182.3 tok/s
• 8-way: 328.1 tok/s aggregate
• 8-way median TTFT: 0.29 seconds

LONG CONTEXT

The engine is configured for the model’s native 262,144-token context.

Actual occupied single-request retrieval testing:

• 122,936 tokens: PASS
• 234,466 tokens: PASS

These were exact retrieval tests, not simply a configured maximum.

The roughly 722K-token KV pool is aggregate capacity across concurrent requests. It is not a 722K single-request context window.

FP8 KV TRADEOFF

FP8 KV gave substantially more cache capacity without a meaningful decode penalty.

The tradeoff is slower long-context prefill. At roughly 131K, TTFT increased from about 87 seconds to about 103 seconds, approximately a 19% penalty.

For a multi-agent workload, I’ve found the extra cache headroom worth that tradeoff.

DFLASH2

Acceptance varies dramatically by workload.

Long free-form prose typically produces an acceptance length around 2.0–2.5.

Across seven days of real agent traffic:

• Average acceptance length: 5.18
• Average acceptance rate: 27.9%
• p90 acceptance length: 7.66
• p99 acceptance length: 10.53

Structured output, tool calls and repetitive code are much easier for the draft model to predict than prose.

TUNING RESULTS

FlashInfer autotune OFF was a major regression. At 2K, decode dropped from roughly 107 to 67 tok/s. At 131K it dropped from roughly 76 to 46 tok/s. I leave autotune ON.

Chunked prefill 8192 provided essentially no decode improvement and increased TTFT. At 65K, TTFT went from about 37 to 42 seconds. At 131K, roughly 104 to 114 seconds. I kept 2048.

Reducing max-running-requests from 8 to 4 increased the available KV pool but provided no useful latency or decode improvement, so I kept 8.

Disabling sleep-on-idle provided no meaningful performance benefit while taking idle CPU usage from approximately 0% to roughly one fully occupied core.

Torch compile does not apply usefully to this DFlashWorkerV2 path in this build.

The speculative token map is also not applicable here because that path is consumed by the EAGLE worker rather than DFlashWorkerV2.

VALIDATION

The build has passed normal chat, coding, reasoning, tool calling, multi-tool calls, structured JSON, streaming, vision, long-context retrieval and 1/2/4/8-way concurrency.

I use /health_generate for end-to-end health testing because it verifies that the engine can actually generate rather than merely confirming that the server process is alive.

PRODUCTION STABILITY

This is probably the most useful data point:

• More than two weeks of production use
• More than 3 billion generated tokens
• Sustained multi-agent concurrent workloads
• Zero container restarts
• Zero OOM events
• No crash loops
• Multiple clean engine boots during tuning
• Idle scheduler CPU approximately 0%
• Startup approximately 3.5 minutes

HOST PINNING

I am deliberately staying on kernel 6.17.0-1032-nvidia, driver 580.173.02, DGX SWBUILD 7.4.0 and OTA 7.5.0.

That combination has been stable under sustained load, so I’m not changing the host stack simply because something newer exists.

The biggest takeaway from tuning this build is that GB10 settings do not necessarily transfer cleanly between recipes. In my case, disabling FlashInfer autotuning and increasing chunked prefill both looked reasonable on paper and both made things worse. Measure changes against the actual workload.

Are you getting better results with the 27B than the Qwen-Next?

Yes. I’ve actually been following Flash-Next pretty closely rather than dismissing it. I’ve been tracking the DGX Spark forum work, Hasso’s SGLang/NEXTN implementation, veloGB10, the various NVFP4/EXL3 builds, long-context tests, tool-calling results and the stability reports as they come in.

Flash-Next is genuinely impressive. Hasso’s current single-Spark NVFP4 lane is around 48 tok/s on code/math and 29–31 tok/s on prose, with vision, prefix caching, 262K context, successful tool tests and exact needle retrieval at 120K/200K. There are also other implementations pushing considerably higher headline decode rates. (GitHub)

What keeps me on 27B+DFlash2 today is the production evidence. My current build has now generated 3B+ tokens over 2+ weeks with sustained multi-agent concurrency, zero OOMs and zero container restarts, and I’ve validated occupied context out to 234K. Flash-Next still has more moving pieces around NEXTN/speculation, PLE/NVMe handling, concurrency tuning and newer serving paths, and I haven’t yet seen the same level of sustained single-Spark multi-agent soak evidence that I have on my current stack.

So I absolutely expect to move to Flash-Next eventually. I’m just waiting for the point where it gives me a clear quality/capability win while being as boring and predictable operationally as the 27B build is now. At that point the switch is easy.

NOTE! This is my primary agent serving LLM. For deeper thinking and coding work I have a TP2 DeepSeek Flash build.