Has anyone tried Qwen 3.8 27B with DFlash2 on a DGX Spark yet?
And this one too, I have been using Mia-Lab’s regular MTP branch, works well.
You will need to change the reasoning effort though. It defaults to xhigh, which is EXTREMELY HIGH thinking
yes, I’m getting 40-42 tok/s
Finally something who gives a real difference, now it’s a little bit more comfortable with Dflash2 :
| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:--------|---------------:|-----------------:|--------------:|-----------------:|-----------------:|-----------------:|
| Acantic | pp2048 | 2989.79 ± 77.34 | | 846.90 ± 30.65 | 637.01 ± 30.65 | 846.90 ± 30.65 |
| Acantic | tg32 | 45.25 ± 7.50 | 46.71 ± 7.75 | | | |
| Acantic | pp2048 | 2980.80 ± 73.20 | | 844.18 ± 28.66 | 634.29 ± 28.66 | 844.18 ± 28.66 |
| Acantic | tg128 | 27.33 ± 1.19 | 27.67 ± 0.94 | | | |
| Acantic | pp2048 @ d4096 | 2067.78 ± 242.06 | | 2872.51 ± 329.41 | 2662.62 ± 329.41 | 2872.51 ± 329.41 |
| Acantic | tg32 @ d4096 | 31.15 ± 11.05 | 32.04 ± 11.49 | | | |
| Acantic | pp2048 @ d4096 | 2043.14 ± 209.36 | | 2990.48 ± 344.56 | 2780.59 ± 344.56 | 2990.48 ± 344.56 |
| Acantic | tg128 @ d4096 | 28.75 ± 1.00 | 29.33 ± 0.94 | | | |
Workload matrix v1 — endpoint http://127.0.0.1:8002 (model id: Acantic)
greedy, fresh single-turn prompts, decode net of prefill (two-call delta)
math (EN, eval-style) : 47.2 tok/s
code (EN) : 40.0 tok/s
code (DE) : 39.1 tok/s
technical explain (FR) : 24.0 tok/s
reasoning (FR) : 46.6 tok/s
free prose (EN) : 21.8 tok/s
free prose (FR) : 20.9 tok/s
free prose (DE) : 18.4 tok/s
Before with best MTP/Dspark config :
| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:--------|---------------:|-----------------:|-------------:|----------------:|----------------:|----------------:|
| Acantic | pp2048 | 3004.54 ± 146.02 | | 857.00 ± 36.05 | 644.76 ± 36.05 | 857.00 ± 36.05 |
| Acantic | tg32 | 25.85 ± 1.20 | 26.67 ± 0.94 | | | |
| Acantic | pp2048 | 3090.33 ± 83.71 | | 816.45 ± 31.32 | 604.20 ± 31.32 | 816.45 ± 31.32 |
| Acantic | tg128 | 22.54 ± 1.52 | 23.00 ± 1.63 | | | |
| Acantic | pp2048 @ d4096 | 2676.93 ± 9.34 | | 2294.11 ± 55.94 | 2081.86 ± 55.94 | 2294.11 ± 55.94 |
| Acantic | tg32 @ d4096 | 22.14 ± 3.63 | 22.67 ± 3.77 | | | |
| Acantic | pp2048 @ d4096 | 2662.70 ± 6.62 | | 2300.43 ± 29.10 | 2088.18 ± 29.10 | 2300.43 ± 29.10 |
| Acantic | tg128 @ d4096 | 23.99 ± 3.07 | 24.33 ± 3.09 | | | |
math (EN, eval-style) : 38.8 tok/s
code (EN) : 31.0 tok/s
code (DE) : 27.0 tok/s
technical explain (FR) : 23.0 tok/s
reasoning (FR) : 31.4 tok/s
free prose (EN) : 16.1 tok/s
free prose (FR) : 15.8 tok/s
free prose (DE) : 13.4 tok/s
I tried applying the path using eugr vllm but it’s complaining about “hf_overrides must be a dict for get_quant_config to get the quantization config from it.” I’ll try rebuilding the container again from scratch and applying the pr again, maybe I missed something
How does DFlash2 performance scale compared to MTP? At high context and/or high concurrency, are you guys seeing comparable performance for both PP and TG?
Currently waiting for mainline SGLang release to have DFlash2 support incorporated, with qwen38 support as well. There are some workarounds where you will have to build your own docker image. I would rather wait for main of SGLang to catch up and use the official docker image.
Currently using MTP, seems fine. But surely the default xhigh thinking effort is truly insane