Just want to share my experience and say a huge thank you to Albond for this technique.
When I got my GX10 in late March, I was stuck at ~30 t/s on Qwen3.5-35B with Ollama. Spent over a week fighting sglang and vLLM with no luck, eventually had to manually compile llama.cpp from source just to hit 72 t/s. For a 128GB machine it felt like leaving most of the hardware on the table.
Albond’s hybrid INT4+FP8 method changed everything. I adapted it for the 35B model, added MTP speculative decoding weights, and here’s what I’m seeing now:
Qwen3.5-35B-A3B — Hybrid INT4+FP8 + MTP (sequential, single request):
| Task | Tokens | Time | Speed |
|---|---|---|---|
| Q&A | 256 | 2.14s | 119 tok/s |
| Code | 453 | 3.47s | 130 tok/s |
| JSON | 1024 | 8.05s | 127 tok/s |
| Math | 32 | 0.29s | 108 tok/s |
| Long code | 2048 | 15.54s | 131 tok/s |
4 concurrent requests: 158.7 tok/s total throughput.
That’s jumping from 72 t/s (manual llama.cpp) to 112–131 t/s depending on task. On 122B: 30 t/s → 51 t/s. Real numbers, no tricks.
I gave Albond’s repo 10 stars (by starring and unstarring repeatedly — I know 😅). Genuinely one of the most impactful things I’ve found for local inference on this hardware.
P.S. If anyone wants to replicate this without digging into the internals, I wrapped the entire pipeline — hybrid checkpoint build, MTP weights, Docker launch, and benchmark — into a single interactive shell script. Just run it and follow the menu. Repo: https://github.com/phuongncn/asus-gx10-qwen35-speed-hack