Benchmarking Qwen3.6-35B-A3B with dspark speculative decoding

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

Hello, I benchmarked Qwen3.6-35B-A3B using the Qwen3.6-35B-A3B-speculator.dspark speculative decoding model.

Environment

uv venv .vllm --python 3.12
source .vllm/bin/activate

uv pip install -U vllm --pre \
  --extra-index-url https://wheels.vllm.ai/nightly/cu130 \
  --extra-index-url https://download.pytorch.org/whl/cu130 \
  --index-strategy unsafe-best-match

2xNVIDIA RTX PRO 6000 Blackwell Max-Q

vllm serve Qwen/Qwen3.6-35B-A3B \
    --tensor-parallel-size 2 \
    --speculative-config '{
        "model": "RedHatAI/Qwen3.6-35B-A3B-speculator.dspark",
        "num_speculative_tokens": 8,
        "method": "dspark"
    }'

Since DGX Spark uses unified memory, I limited the number of compilation jobs with MAX_JOBS=2 to reduce RAM usage:

MAX_JOBS=2 vllm serve Qwen/Qwen3.6-35B-A3B \
   --tensor-parallel-size 1 \
   --speculative-config '{
       "model": "RedHatAI/Qwen3.6-35B-A3B-speculator.dspark",
       "num_speculative_tokens": 8,
       "method": "dspark"
   }'

Results

 Scenario                    | Vanilla |  k=1 |  k=2 |  k=4 |  k=8
 ----------------------------+---------+------+------+------+-----
 Prompt-heavy (8k in/1k out) |   1766  | 1249 | 1218 | 1131 |  777
 Decode-heavy (1k in/8k out) |   3158  | 2080 | 2099 | 1978 | 1475
 Balanced    (1k in/1k out)  |   3193  | 2646 | 2507 | 2144 | 1536
2

can you share more details about your test tool? such as what is vanilla? and what’s the unit of these numbers 1766 | 1249 | 1218? how do you measure prefill and decode performance via these numbers?

3

First of all - thanks for this research. Have you directly compared the values ​​with the built-in MTP version? It would be great if you tested your recipe using one of the popular benchmarking tools used on this forum for direct comparisons with other solutions.

4

I used a random dataset with the vllm benchmark. On a hard reasoning problem, the acceptance rate will be quite low, maybe around 25%. But on a more structured and straightforward task, such as using a web search tool or coding, it could be around 70% or even higher.