So it’s looking like this might be the best local model we’re likely to get in 2026 (EDIT: runnable by people who aren’t oil sheiks), except maybe Qwen 3.6 122B. I’m curious the max inference speed I can get out of it on a Spark. We all know about quantization, so that’s not what I’m asking here. S…

Maybe we start a new thread on quantization because I don’t want to derail this thread, but between Gemini and Claude neither could come up with a way that worked for trying to quantize GLM-5.1 (1.5TB model) for me into AWQ. I was trying to do this on a rig with an RTX 6000 and also 96GB system RAM,…

I was curious about this too, Prismquant or otherwise, because I have 4 Sparks and that would give me about 4 x 117GB to work with. Again, maybe we start a new self-quantization thread, though.

I just want to thank you all on this thread. I put these changes in place and was amazed to get a 2.66x improvement in tok/s with the MTP changes. My G*d. This is more like what I had hoped from GX10… Just wanted to say thanks! You helped a tinkerer learn something.

I haven’t seen something that needed it yet. Honestly, not sure!

I have an LRU cache and cache prefetching implemented to dynamically load and unload things from disk. Prismquant needs to read all the layers because it does a quite complex determination of weight-importance and it has to traverse the whole model to do that. I find AI tools are amazing but not ne…

Speculative decoding is underdocumented in general. It’s a free speed boost. It should be the first thing every guide mentions.There’s also different techniques all called speculative decoding. MTP is the SOTA, for models that support it. But many don’t, including Gemma 4. Only ones I know support …

In case anyone is still interested in this - I operate a cluster of 2 GX10s (Asus’s version of the DGX SPARK) and I got this from Unsloth’s NVFP4: $ .venv/bin/llama-benchy --base-url http://localhost:8081/v1 --model unsloth/Qwen3.6-27B-NVFP4 --tokenizer /home/coder/.cache/huggingface/hub/models–u…

[image] ttsiodras: 32 t/s is 32 t/s with concurrency? I only get 14-15 t/s sorry, edit: I used only one Spark, not two

On a single-node Asus GX10, AEON-Ultimate-Uncensored-DFlash , for comparison: (llama-benchy) redacted@cabana:~/llama-benchy$ llama-benchy --base-url http://10.67.0.156:8000/v1 --model qwen --depth 0 4096 8192 16384 32768 --latency-mode generation --tokenizer /home/redacted/Qwen3.6-27B-AEON-Ultimate…

For my current work behavior (analyzing a lot of different code from different projects, and improving it) I found sweetspot from combining prebuilt docker image with stable vllm 0.20 with dflash support from AEON and rdtand/Qwen3.6-27B-PrismaQuant-5.5bit-vllm version of the LLM. Here is my ultimat…

What's the best speed we can get with Qwen 3.6 27B without quantizing?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

garlatiromeo June 13, 2026, 9:32pm 33

Thank you very much! This is finally useful , works at great performance similar perf. to qwen3.5-122b-10a, but it is a much better model for coding.

Topic		Replies	Views
Qwen/Qwen3.6-35B-A3B (and FP8) has landed DGX Spark / GB10 agentic-ai	309	27676	June 22, 2026
Qwen3.6-27B is out! DGX Spark / GB10 agentic-ai	297	27433	June 16, 2026
Qwen3.5 27B optimisation thread starting at 30+ t/s TP=1 DGX Spark / GB10 llama , agentic-ai	23	2837	May 11, 2026
Qwen3.5-122B-A10B on single Spark: up to 51 tok/s (v2.1 — patches + quick-start + benchmark) DGX Spark / GB10 cuda , performance , docker , performance-tuning , llm	434	22167	June 24, 2026
Qwen3.5-122B-A10B NVFP4 Quantized for DGX Spark — 234GB → 75GB, Runs on 128GB DGX Spark / GB10 Projects	44	11429	April 9, 2026
Bfloat16 Quality = Speed? DGX Spark / GB10	106	5981	May 26, 2026
Qwen/Qwen3.5-122B-A10B - Alibaba/Qwen thought about us... :-D DGX Spark / GB10	340	17000	March 24, 2026
Fastest Qwen 3.5 122B Int4 recipe on DGX Spark tested and published on Spark-Arena DGX Spark / GB10 llama	59	3126	June 3, 2026
Benchmark Report: Qwen3.6-35B-A3B-NVFP4 on NVIDIA DGX Spark, Jetson Thor, Blackwell 6000 Pro DGX Spark / GB10 Projects	10	2864	June 2, 2026
HOW-TO: Run Qwen3-Coder-Next on Spark DGX Spark / GB10 llama	92	10292	March 24, 2026

What's the best speed we can get with Qwen 3.6 27B without quantizing?

Related topics