Degenerate logits at temperature > 0 — random CJK characters spliced into English output (vLLM, DeepSeek-V4-Flash, TP=2)

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Degenerate logits at temperature > 0: single CJK characters randomly spliced into English output (2× DGX Spark, GB10)

Environment

  • Hardware: 2× NVIDIA DGX Spark (GB10, sm_121, 128GB unified memory), TP=2 across nodes via 200GbE RoCE direct connect
  • Engine: vLLM 0.25.2 (image ghcr.io/anemll/dspark-vllm-gx10:0.1.1, digest-verified)
  • Model: DeepSeek-V4-Flash-Vision-Exp, official FP8 weights (48 shards)
  • Key params: DSpark speculative decoding (MTP=6), --moe-backend flashinfer_b12x, --kv-cache-dtype nvfp4_ds_mla,
    --max-model-len 1048576, --max-num-seqs 6, --async-scheduling, prefix caching on

Problem

At any temperature > 0, English output sporadically contains a single CJK character
spliced into words (e.g. compile玄, verify纲). The inserted token’s logprob equals
-ln(vocab) ≈ -11.77, i.e. the logits at that position are degenerate (uniform/uninitialized).

  • temp = 0: completely clean (0/85 samples) — argmax masks the degenerate logits
  • temp = 0.2–0.7: intermittent; ~0.8% on short prompts, up to 3–8 out of 10 responses
    on long agent-style prompts (hundreds of rules + code analysis)

Already ruled out (controlled experiments)

  • Speculative decoding (disabled → still reproduces)
  • Temperature scaling path (temp 1.0 → worse, 8/10)
  • Engine version (also reproduces on an older vLLM 0.21.1-based build)
  • sparse-SWA RoPE fix (vllm#54815) → no effect

Suspect

Model forward/logits path correctness bug specific to GB10 (sm_121) — not a sampling
parameter issue. Workaround: force temperature 0 (deterministic output; reasoning via
reasoning_effort still works normally).

2

Some vLLM versions have accuracy issues with the b12x backend for this model especially under concurrency. Here’s what I’ve been using:

It performs really well, and I haven’t found any degeneracy issues even at concurrent high KV usage. If you’re willing to switch stacks you can point an agent at the repo and it should be able to get everything set up.

3

I have some fixes for deterministic output in my tree.

Point your agent at it to see if it finds anything useful?