Word of warning for Tony’s repo: Despite the name, the default model the repo uses is the OG V4 Flash model, so that could also be the cause of hallucinations. I had the same experience as you before I realized that, and I swapped the model out. The DSpark model it uses is based on the older V4 Flash, and 0731 includes DSpark, so you can just swap the model in the docker-compose file.
As for Qwen 3.8 Flash, here’s my config. It’s NVFP4 experts, FP8 N-gram table. I know this thread is about FP8, but I’ve been having a good experience with it so far. Getting roughly 35-40 tokens per second with it. Qwen 3.8 Flash - 2x DGX Spark · GitHub
Both Sparks also run at 2000MHz memory clock.
tool-eval-bench:
✓ Warm-up complete (236 ms)
🔍 Engine: vLLM 0.1.dev20073+g8e685d198
╭─────────────────────────────────────────────────────────────────────────── ⚡ llama-benchy Throughput Benchmark ────────────────────────────────────────────────────────────────────────────╮
│ RadixArk/Qwen3.8-Flash-Next-NVFP4 │
│ pp=[2048] tg=[128] depth=[0, 4096, 8192] concurrency=[1, 2, 4] runs=3 latency=generation │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
✓ Complete ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27/27 0:04:23
llama-benchy 0.4.0
Estimated latency: 251.0 ms
llama-benchy Results
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test ┃ c ┃ pp t/s ┃ tg t/s ┃ TTFT (ms) ┃ Total (ms) ┃ Tokens ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ pp2048 tg128 @ d0 │ c1 │ 3,249 │ 37.6 │ 898 │ 4,054 │ 2048+128 │
│ pp2048 tg128 @ d0 │ c2 │ 2,160 │ 56.5 │ 1,606 │ 5,443 │ 2048+128 │
│ pp2048 tg128 @ d0 │ c4 │ 2,438 │ 85.2 │ 3,119 │ 7,446 │ 2048+128 │
│ pp2048 tg128 @ d4096 │ c1 │ 3,005 │ 35.3 │ 2,335 │ 5,711 │ 2048+128 │
│ pp2048 tg128 @ d4096 │ c2 │ 2,822 │ 42.3 │ 3,294 │ 7,615 │ 2048+128 │
│ pp2048 tg128 @ d4096 │ c4 │ 2,513 │ 40.0 │ 7,861 │ 13,427 │ 2048+128 │
│ pp2048 tg128 @ d8192 │ c1 │ 2,646 │ 35.1 │ 4,297 │ 7,690 │ 2048+128 │
│ pp2048 tg128 @ d8192 │ c2 │ 2,633 │ 52.4 │ 7,380 │ 11,186 │ 2048+128 │
│ pp2048 tg128 @ d8192 │ c4 │ 2,748 │ 36.9 │ 12,142 │ 18,339 │ 2048+128 │
└───────────────────────────────────────────────┴─────────────┴────────────────────────┴────────────────────────┴─────────────────────────┴─────────────────────────┴─────────────────────────┘
ℹ Metrics sourced from llama-benchy — see https://github.com/eugr/llama-benchy for methodology.