⚡ Update: v2 (post #71) achieves 51 tok/s. v2.1 (post #104) adds a quick-start script. See those posts for the latest setup. Been chasing every last token/second out of Qwen3.5-122B-A10B on a single DGX Spark for the past few weeks. Not sure if anyone else is still optimizing this model on Spark, b…

Did you come across errors like this? WARNING: unexpected unmatched FP8 tensor model.language_model.layers.0.linear_attn.in_proj_a.weight WARNING: unexpected unmatched FP8 tensor model.language_model.layers.0.linear_attn.in_proj_b.weight WARNING: unexpected unmatched FP8 tensor model.language_mo…

Thanks for sharing. I’ll give it a try … and feedback Working mainly with the 35B version, but would love to switch to 122B, if only the speed would increase. Your solution looks promising.

If I understand your investigation correctly, to be able to benefit from MTP, we only have to build a new model version with: python patches/02-mtp-speculative/add-mtp-weights.py \ --source "$INTEL_DIR" \ --target ~/models/qwen35-122b-hybrid-int4fp8 and all those you simply added the line …

Yes, this is expected. The 408 unmatched FP8 tensors are mostly linear_attn projections (DeltaNet layers — 36 out of 48 layers in Qwen3.5) plus some attention norms and gates. They exist in the Qwen FP8 checkpoint but don’t have matching counterparts in the Intel AutoRound INT4 checkpoint because th…

yes, MTP is the bigger win, and it works independently of the hybrid patch. I haven’t actually benchmarked MTP on the plain Intel AutoRound INT4 checkpoint without the hybrid patch. What I did test end-to-end: Baseline INT4 + FlashInfer: 28.3 tok/s (verified) Hybrid INT4+FP8 + FlashInfer: 30.8 t…

The whole point of this optimization work was to get Claude-level intelligence without Claude-level costs. Qwen3.5-122B scores 42 on the Artificial Analysis Intelligence Index — one point below Claude 4.5 Sonnet (43), and beats it on IFBench (76% vs 57%) and Humanity’s Last Exam (23% vs 17%). That’…

Hybrid INT4+FP8: detected 144 FP8 dense layers (block_size=[128, 128]) Sounds good :D currently evaluating, as I am running this on the community docker, which is only vLLM 0.18.1rc1.dev41. But seems to work, thank you! ── Run 1/2 ────────────────────────────────────── [Q&A] 256 tokens in 6.62…

The script doesn’t seem to be doing anything for me: Found 785 MTP tensors in source index Added 0 MTP tensor mappings to index Total tensors: 112901 Done. MTP speculative decoding is now available.

About the llama-benchy numbers — the difference is real, just measured differently. Think of it this way: without MTP, the model does 1 decode step = 1 token. With MTP, the model does 1 decode step but produces ~2 tokens (1 regular + 1 speculative, 95% accepted). llama-benchy measures decode steps…

Qwen3.5-122B-A10B on single Spark: up to 51 tok/s (v2.1 — patches + quick-start + benchmark)

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

norman.2 April 5, 2026, 10:00am 2

As the patches are python only, we can probably integrate this as mod @eugr ? :)

Nice thank you for the effort! Will give this a try, as this model is usually my daily driver.

3 Likes

Topic		Replies	Views
Qwen/Qwen3.5-122B-A10B - Alibaba/Qwen thought about us... :-D DGX Spark / GB10	340	14369	March 24, 2026
Does Qwen3.5-35B-A3B on GB10 leave a lot of performance on the table? DGX Spark / GB10 agentic-ai	40	4434	March 16, 2026
Qwen3.5-122B-A10B NVFP4 Quantized for DGX Spark — 234GB → 75GB, Runs on 128GB DGX Spark / GB10 Projects	44	7995	April 9, 2026
Qwen3.5-122B-A10B on single Spark: 15 → 21.5 tok/s with hybrid GPTQ-INT4 + FP8 dense layers (https://github.com/rmstxrx/vllm-hybrid-quant) DGX Spark / GB10 cuda	9	648	March 20, 2026
Qwen3.5-397B-A17B run in dual spark! but I have a concern DGX Spark / GB10	216	5752	April 11, 2026
HOW-TO: Run Qwen3-Coder-Next on Spark DGX Spark / GB10 llama	92	8265	March 24, 2026
RedHatAI/Qwen3.5-122B-A10B-NVFP4 seems to be the best option for a single Spark DGX Spark / GB10 Projects llm	74	4183	April 11, 2026
Qwen3.5-397B-A17B + DGX Spark (duo) DGX Spark / GB10 Projects	55	4287	April 6, 2026
Qwen3.5 27B optimisation thread starting at 30+ t/s TP=1 DGX Spark / GB10 llama , agentic-ai	15	664	April 12, 2026
50%+ Improvement on spark?! DGX Spark / GB10 cuda , deepseek	26	1793	April 7, 2026

Qwen3.5-122B-A10B on single Spark: up to 51 tok/s (v2.1 — patches + quick-start + benchmark)

Related topics