Qwen3.8-Flash-Next, GLM-5.3-Flash, DeepSeek-V4-Flash on 2x DGX Spark - prefill, TTFT, SWE-bench Pro

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Three models serving TP=2 on the same pair of DGX Sparks, same benchmark scripts. Prefill and TTFT first, one full-length request, 5 needles, token counts verified via /tokenize:

context    Qwen3.8-FN FP8    GLM-5.3 NVFP4    DeepSeek-V4 NVFP4
             ttft  prefill    ttft  prefill     ttft  prefill
  4,096     3.40s  1206      2.54s  1615       3.88s  1057
 32,768    15.93s  2057     20.79s  1576      27.28s  1201
~131,072   50.60s  2591     66.71s  1965     114.65s  1134

Prefill in tok/s. Qwen’s prefill rate rises with context, 1206 to 2591. GLM and DeepSeek stay flat. At 131K that is Qwen at 50s against DeepSeek at 115s for the same prompt.

Needles 5 of 5 for all three at every depth. Max verified: GLM 131K, Qwen 245K, DeepSeek 265,781.

Single-stream decode, each at its own serving config:

Qwen3.8-FN NVFP4   29.6 tok/s   262K, no spec, seqs 64
Qwen3.8-FN FP8     23.0 tok/s   262K, no spec, seqs 64
GLM-5.3 NVFP4      21.5 tok/s   MTP-4, seqs 6
DeepSeek-V4 NVFP4  41.0 tok/s   1M, DSpark spec, seqs 48

SWE-bench Pro, 40-instance enterprise subset, same harness:

Qwen3.8-Flash-Next   36/40   FP8 and NVFP4, 262K
Qwen3.8-Flash-Next   38/40   NVFP4, NVIDIA checkpoint, 1M YaRN, MTP-1
GLM-5.3-Flash        36/40   NVFP4, 262K

DeepSeek-V4-Flash I have not run on SWE-bench Pro.

Full configs and raw json per model:

2

On MTP depth for the Qwen3.8-Flash-Next NVFP4 builds, since the two published checkpoints differ here.

The NVIDIA checkpoint block-quantizes its MTP head to FP8, which the modelopt path in vLLM does not load (w2_weight_scale_inv AttributeError). It needs PR 55513, still unmerged, ported in. Once loaded I swept the draft depth:

config              c1 agg   c4 agg   c16 agg
NVIDIA MTP off       21.88    72.93    215.01
NVIDIA MTP k=1       31.44    87.92    215.67
NVIDIA MTP k=3       31.07    82.75    202.65

k=1 wins on every axis, and k=3 at c16 is below no MTP at all. Deeper speculation loses once batching already saturates the machine. k=1 is what NVIDIA’s model card specifies.

The RadixArk checkpoint keeps its MTP head in a format vLLM loads natively, no patch, and I ran it at k=3:

config              c1 agg   c4 agg   c16 agg
RadixArk MTP k=3     29.61    92.42    212.09
NVIDIA  MTP k=3      31.07    82.75    202.65

Same draft depth, NVIDIA is faster single stream but RadixArk pulls ahead under concurrency, c4 92 vs 83 and c16 212 vs 203. A real crossover, not noise.

So on NVIDIA use k=1, and if throughput is the only axis the two checkpoints are within about 6 percent once both run MTP. The reason to prefer the NVIDIA build is the calibration set, not the tok/s.

3

DeepSeek number is in. 33/40 on the same 40 instances.

Qwen3.8-Flash-Next NVFP4 (NVIDIA, 1M, MTP-1)  38/40
Qwen3.8-Flash-Next FP8                        36/40
GLM-5.3-Flash NVFP4                           36/40
DeepSeek-V4-Flash-0731 MXFP4                  33/40

Worth knowing before you run DeepSeek on vLLM: it scored 22/40 the first time, and that was my launch line, not the model. vllm/tokenizers/deepseek_v4.py reads

thinking = kwargs.get("thinking", False)

so with no --default-chat-template-kwargs the encoder runs thinking_mode “chat” and emits the thinking end token immediately. Reasoning is off, nothing warns you, and the output still reads fine.

The trajectories show it better than the score does:

                        reasoning off   reasoning on
submitted with a patch      34/40          40/40
LimitsExceeded                  3              0
RepeatedFormatError             3              0

The format errors were not a parser problem. The same instances parse cleanly once the model can think.

DeepSeek’s own Code Agent recipe is reasoning_effort max at temperature 1.0, top_p 0.95:

--default-chat-template-kwargs '{"thinking":true,"reasoning_effort":"max"}'
--override-generation-config '{"temperature":1.0,"top_p":0.95}'

The override matters too. --generation-config vllm makes vLLM ignore the checkpoint’s own generation_config.json, and that file is a generic stub anyway.

Check reasoning_content comes back non-empty on one request before trusting any run. Zero means reasoning is off whatever the launch line looks like.

This row is MXFP4 with speculation off, so do not read it against the 41 tok/s in the first post. At max effort it is also the slowest of the three here, about 5.7 hours for the 40 against roughly 2 for Qwen.