(MiaAI_lab) New Qwen3.8-Flash-Next NVFP4 recipe for 1× DGX Spark — 1M context, vision/video, 37 tok/s C1

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

This looks especially interesting for people using a single DGX Spark for Hermes, OpenCode, browser agents and other multimodal workloads.

Reported results:

  • up to 1M context
  • 1,431,164 KV cache
  • full image + video support
  • around 37 tok/s single-stream prose decode
  • up to 86 tok/s aggregate with 4 concurrent streams
  • around 1500–2000 tok/s prefill
  • stress-tested with a 400K prompt

What makes this especially interesting is the combination of large context + multimodal support + decent concurrency on just one Spark. For browser/agent workloads, this looks like one of the most practical Qwen3.8-Flash-Next setups available right now.

GitHub recipe:
MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark

Model checkpoint:
Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 on Hugging Face

MiaAI:
@MiaAI_lab on X

I’m planning to test this with Hermes Desktop as well. Would be interesting to see results from other single-Spark owners, especially for long-context browser agents and multimodal tool use.

image
image774×973 120 KB

Technical results

MiaAI also published detailed memory, KV-cache and long-context measurements for the single-Spark setup.

KV cache / long-context profiles

Configuration KV pool Prefill @ 400K Decode, prose idle Needle tests 5/50/95%
262K, KV_TARGET_GIB=20, BF16 21.28 GiB = 736,837 tokens, ~2.81× 262K requests — — —
512K YaRN, KV_TARGET_GIB=20, BF16 19.2 GiB = 704,558 tokens, ~1.34× 512K requests 1,537 tok/s, TTFT 260.3 s 28.3 tok/s ±2.6 3/3 PASS
512K YaRN, KV_TARGET_GIB=22, BF16 796,196 tokens, ~1.52× 512K requests 1,883 tok/s @ 32K — 12/14
512K YaRN, KV_TARGET_GIB=22, FP8 22.2 GiB = 1,431,164 tokens, ~2.73× 512K requests 1,495 tok/s @ 400K, TTFT 267.7 s; 1,769 tok/s @ 32K 27.1 tok/s ±1.3 15/20

The most interesting number here is the 1.43M-token FP8 KV pool on a single DGX Spark. That gives a lot of headroom for multiple long-running agent sessions, although the authors now recommend a more conservative host-memory-limited default profile.

Current shipped profile — measured 2026-09-05

Current default profile:

262K context, HOST_RESERVE_GIB=26, KV_TARGET_GIB=16, FP8 KV, MAX_NUM_SEQS=5

Metric Result
GPU memory budget GMU 0.780 = 94.87 GiB
Available KV cache 16.46 GiB = 992,584 tokens, ~3.79× 262K requests
Time until /health 10 min 51 sec from NVMe checkpoint load
Host MemAvailable, 2 min after /health 15.7 GiB, MemFree 5.1 GiB
Host MemAvailable, after 40 min idle 15.5–16.4 GiB, MemFree ≥4.4 GiB
After two ~90K-character requests 16.2 → 15.05 GiB after first; ~15.2 GiB after 60 s after second; minimum ~14.9 GiB during prefill
Five concurrent ~60K requests 14.9 → 14.57 GiB after +60 s; minimum 14.26 GiB; 5/5 completed, no watchdog events
2.5 h qwen-code workload ~38 requests, 19 at 50–100K tokens, up to 3 concurrent; MemAvailable 14.2–14.9 GiB between turns, minimum 12.8 GiB
NV_ERR_NO_MEMORY in journalctl -k 0 during startup and all tests above

This is important because earlier more aggressive KV_TARGET_GIB=20/22 profiles left only around 6.9–8.8 GiB MemAvailable, very close to the point where the GB10 driver starts refusing allocations.

The current MiaAI recipe therefore uses a host-side safety budget through HOST_RESERVE_GIB, rather than simply pushing GPU_MEMORY_UTILIZATION as high as possible.

Safety notes:

Prefill throughput

Prefill was measured using sparkDash:

The two columns are not a strict A/B test because they use different context profiles, KV targets and batch widths, but they show how much MAX_NUM_BATCHED_TOKENS=8192 helps amortize per-block overhead.

Context 2048-token batches — 512K YaRN, KV_TARGET_GIB=22 8192-token batches — 262K native, KV_TARGET_GIB=20
8K 5.00 s · 1,646 tok/s 3.69 s · 2,228 tok/s
16K 8.00 s · 2,052 tok/s 7.16 s · 2,293 tok/s
32K 15.83 s · 2,073 tok/s 13.87 s · 2,366 tok/s
64K 32.20 s · 2,037 tok/s 28.31 s · 2,316 tok/s
128K 67.41 s · 1,945 tok/s 58.88 s · 2,227 tok/s
256K 146.40 s · 1,791 tok/s not re-measured

The 8192-token batching profile is especially interesting for agent workloads because prefill remains above roughly 2.2K tok/s all the way to 128K context.

Source / recipe:

sparkDash:

2

Benchmark: tool-eval-bench v2.6.1.dev45+gcf54b4bfe

image
image2536×866 130 KB

3

Was this better that the blazux repo for you? I have a similar use case to yours and also use Hermes desktop. I am liking Qwen 3.8 Flash Next vs the tpurtell Deepseek variant. It just seems to work its way around problems.

4

I have extensively used the qwen and deepseek on by dual sparks. qwen consistently writes double the amount of code and introduces 3x the security flaws. it does write the code more “elegantly” (aka not the fastest implementation but its properly layered…or bloated, depending on how you like your code). But the security flaws are a no no for me. It makes way too many errors in that area. I have removed it from circulation and will not be coming back.

It is better than the 27B in some ways, but not in others. ds4f beats it handily.

5

i was trying Mia’s recipe. Speed wise it’s brilliant, but it looks like quantization degraded model quite a lot. I am staying with GitHub - blazux/qwen3.8-Flash-DGX: recipe for running Qwen3.8-Flash-Next on a single DGX Spark · GitHub

6

I tried the MiaAI-Lab recipe as well, and unfortunately I came away pretty disappointed with the actual model quality.

From an engineering perspective, the recipe is impressive: high throughput, a huge KV cache, very long context, multimodal support, and aggressive memory optimization for a single DGX Spark.

But after actually using it, the difference in response quality compared with blazux/qwen3.8-Flash-DGX became quite noticeable to me.

My main use cases are Hermes Desktop, coding agents, reasoning, browser/tool use, long conversations, and multi-step agent tasks. In these scenarios, the Mia version feels significantly more degraded to me. Responses are less reliable, reasoning feels less consistent, and overall model behavior is noticeably worse than with the blazux recipe.

This is roughly how I would visualize the trade-off based on my own testing and experience. This is not a scientific benchmark, just a simplified representation of what I am seeing in real use:

                         QUALITY     SPEED       MEMORY EFFICIENCY

blazux NVFP4             ██████████  ██████      █████
blazux hybrid            █████████▌  ████████▌   ████████
MiaAI-Lab                ██████      ██████████  ██████████

blazux NVFP4 — my choice if preserving as much of the original model quality as possible is the priority. It is the most conservative configuration here, keeping more sensitive parts of the model at higher precision. The trade-off is lower speed and less memory available for KV cache.

blazux hybrid — in my opinion, the best overall balance for a single Spark. It is noticeably faster and more memory-efficient than pure NVFP4 while retaining almost the same quality in practice. The author also reported the same 45/51 vs 45/51 result for NVFP4 and hybrid in their agentic test.

MiaAI-Lab — technically the most aggressive of these three in terms of maximizing throughput, KV capacity, and usable context on one Spark. The engineering behind it is genuinely impressive, but more parts of the model are compressed/optimized, and in my own real-world testing the quality trade-off is clearly noticeable.

So for actual model quality, my current ranking is:

  1. blazux MODE=nvfp4
  2. blazux MODE=hybrid
  3. MiaAI-Lab

For everyday use with Hermes, coding agents, browser agents, and long tool loops, I would personally choose blazux hybrid. I would rather give up some throughput and KV capacity than lose model quality and consistency across a long agent session.

For me, the benchmark numbers from Mia look excellent, but the actual model feels substantially more degraded in real use.

There are also faster Qwen3.8-Flash-Next recipes on the NVIDIA Forum, such as azampatti/GB10-3.8-Flash-Next, albond/SingleSpark-Qwen3.8-Flash-Next, and the HashK-PLE / flashnext-one-spark approach.

I didn’t include them in this comparison because I was specifically looking at configurations with the largest practical context window, as my main use case is agentic workloads.

7

Same experience here. I run blazux hybrid on a single Spark through Hermes Desktop and it holds up a lot better on long agent sessions than the fp8 KV builds. Tried the higher-context route too and the quality drop wasn’t worth it for me. Thank for validating for a similar setup. I do keep chasing speed and enjoy experimenting with new recipes and models.