Sharing my single-DGX-Spark Qwen3.8-27B configuration after more than two weeks of production use, more than 3 billion generated tokens, and sustained multi-agent concurrency.
This is the stable configuration I’ve ended up with after testing quite a few GB10 tuning combinations.
HOST
• NVIDIA DGX Spark / GB10
• Ubuntu 24.04.3
• Kernel: 6.17.0-1032-nvidia
• NVIDIA driver: 580.173.02
• CUDA 13.0
• DGX SWBUILD 7.4.0 + OTA 7.5.0
MODEL
• Target: RadixArk/Qwen3.8-27B-NVFP4
• Revision: 554ebba9b5f1b79dc11246341960360e6ef05ef4
• NVFP4 mixed precision
• Native context: 262,144
• Vision supported
DFlash2 draft:
• maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal
• Revision: bd7a934213c47a9e7ef69eef36bb3325f47fd1f1
• 16 draft tokens
• 1 speculative step
• selector_rank=256
• selector_top_k=16
• speculative_eagle_topk=1
RUNTIME
• SGLang 0.0.0.dev0+qwen38.27b.g561c8f3
• Commit c4271c3fe1262fc2adbd162c33b25de5255251c5
• DFlash2 PRs #35371 and #35496
• PyTorch 2.13.0+cu130
• Triton 3.7.1
• FlashInfer 0.6.18
SERVING CONFIGURATION
• Context: 262,144
• KV cache: fp8_e4m3
• mem-fraction-static: 0.65
• Chunked prefill: 2048
• Max prefill tokens: 16,384
• Max running requests: 8
• Attention backend: FlashInfer
• Mamba SSM: bfloat16
• Mamba radix strategy: extra_buffer
• mamba-full-memory-ratio: 11.93
• DFlash2 draft tokens: 16
• FlashInfer autotune: ON
• Sleep-on-idle: ON
• Prefill CUDA graph: OFF
• Continuous decode steps: 1
• Torch compile: OFF
• Speculative token map: not used
The resulting KV pool is approximately 722K tokens, with roughly 40 GB still available after startup and cache allocation.
CONTROLLED PERFORMANCE
On a quiet engine with the stock target:
• ~2K context: 106 tok/s
• ~65K: 92 tok/s
• ~131K: 85 tok/s
• Long free-form prose: typically 22–24 tok/s
Concurrency testing:
• 1-way: 58.5 tok/s aggregate
• 2-way: 92.3 tok/s
• 4-way: 182.3 tok/s
• 8-way: 328.1 tok/s aggregate
• 8-way median TTFT: 0.29 seconds
LONG CONTEXT
The engine is configured for the model’s native 262,144-token context.
Actual occupied single-request retrieval testing:
• 122,936 tokens: PASS
• 234,466 tokens: PASS
These were exact retrieval tests, not simply a configured maximum.
The roughly 722K-token KV pool is aggregate capacity across concurrent requests. It is not a 722K single-request context window.
FP8 KV TRADEOFF
FP8 KV gave substantially more cache capacity without a meaningful decode penalty.
The tradeoff is slower long-context prefill. At roughly 131K, TTFT increased from about 87 seconds to about 103 seconds, approximately a 19% penalty.
For a multi-agent workload, I’ve found the extra cache headroom worth that tradeoff.
DFLASH2
Acceptance varies dramatically by workload.
Long free-form prose typically produces an acceptance length around 2.0–2.5.
Across seven days of real agent traffic:
• Average acceptance length: 5.18
• Average acceptance rate: 27.9%
• p90 acceptance length: 7.66
• p99 acceptance length: 10.53
Structured output, tool calls and repetitive code are much easier for the draft model to predict than prose.
TUNING RESULTS
FlashInfer autotune OFF was a major regression. At 2K, decode dropped from roughly 107 to 67 tok/s. At 131K it dropped from roughly 76 to 46 tok/s. I leave autotune ON.
Chunked prefill 8192 provided essentially no decode improvement and increased TTFT. At 65K, TTFT went from about 37 to 42 seconds. At 131K, roughly 104 to 114 seconds. I kept 2048.
Reducing max-running-requests from 8 to 4 increased the available KV pool but provided no useful latency or decode improvement, so I kept 8.
Disabling sleep-on-idle provided no meaningful performance benefit while taking idle CPU usage from approximately 0% to roughly one fully occupied core.
Torch compile does not apply usefully to this DFlashWorkerV2 path in this build.
The speculative token map is also not applicable here because that path is consumed by the EAGLE worker rather than DFlashWorkerV2.
VALIDATION
The build has passed normal chat, coding, reasoning, tool calling, multi-tool calls, structured JSON, streaming, vision, long-context retrieval and 1/2/4/8-way concurrency.
I use /health_generate for end-to-end health testing because it verifies that the engine can actually generate rather than merely confirming that the server process is alive.
PRODUCTION STABILITY
This is probably the most useful data point:
• More than two weeks of production use
• More than 3 billion generated tokens
• Sustained multi-agent concurrent workloads
• Zero container restarts
• Zero OOM events
• No crash loops
• Multiple clean engine boots during tuning
• Idle scheduler CPU approximately 0%
• Startup approximately 3.5 minutes
HOST PINNING
I am deliberately staying on kernel 6.17.0-1032-nvidia, driver 580.173.02, DGX SWBUILD 7.4.0 and OTA 7.5.0.
That combination has been stable under sustained load, so I’m not changing the host stack simply because something newer exists.
The biggest takeaway from tuning this build is that GB10 settings do not necessarily transfer cleanly between recipes. In my case, disabling FlashInfer autotuning and increasing chunked prefill both looked reasonable on paper and both made things worse. Measure changes against the actual workload.