I’m a big fan of the Qwen 3.5 397B A17B model running across two DGX Spark nodes.
I was a bit disappointed that the recently released MiniMax M3 was simply too large to run comfortably on a two-node setup. However, it looks like a promising post-trained/fine-tuned variant based on Qwen 3.5 397B A17B has now become available.
I’m currently downloading the model to give it a try. Once the download is complete, I plan to quantize it to W4A16 using RTN with intel/AutoRound and then run some tests.
Judging by the benchmark scores alone, it seems to be roughly on par with Qwen 3.7 Plus, but I think the real verdict will only come from hands-on use.
Unfortunately, my home internet connection is pretty slow, so the download is going to take a while. If anyone has already tried this model, I’d love to hear your impressions and how it performs in practice before I get a chance to test it myself.
What I like most is that it supports a 1M-token context window and is released under the MIT license.
Of course, handling the full 1M-token context on just two nodes will probably be unrealistic, but I’m wondering if TurboQuant could make it possible to push the context length a bit further and use significantly more tokens.
In any case, if anyone has experimented with this setup, I’d be very interested to hear about their experience.
Oh, then I’ll rebuild the vLLM image based on this and test it after applying the INT4 RTN calibration-free quantization tomorrow. Thanks for the information!
Unfortunately turbo quant or rotor quant add lot of computation, so you can fit more but at a cost of prefill and decode too. Same as yarn. Once yarn kicks in prefill drops 70% and so is decode. Experienced with qwen models many times.
There’s a bit of, shall we say, drama going on between Rio and Nex where it seems that Rio is a merge of Nex with the original base model with very mild tuning after that.
Rio has changed their Readme to admit this now.
An interesting read:
I’m not sure if the Rio MIT license would hold up under actual challenge. The parents are not MIT and though I’m not a lawyer, I suspect their tuning is not sufficient to drop the prior licenses entirely.
What does then interest me is if Nex N2 could do latent space reasoning…
I tell you, this is the stuff that makes me wonder about everything being claimed by people everyday. Everyone is so focused on marketing versus plain truths. Why not be upfront and say “Oh, we merged this and that and it’s cool!” Self-aggrandisement is not a virtue by any stretch.
regardless, this means they compared it to qwen3.6 plus and it scored as well in some benchmarks. so the nex model may be useful as a refresh to 3.5 after all.
Damn m3 is LAST? Is it because its lobotomized maybe because of improper support currently or because its super slow? Kinda disappointing if the model is very sensitive to quantizing, even more so than m2.7. Edit: I just saw the REAP part ok now that makes more sense.
Tried your toolery tools, thanks you, very cool project, get the point to make it difficult but it delivers very confusing results, not reflecting real usage pattern, imo. Right in your tests qwen3 coder scores high, in reality it is near useless for modern agentic work, which it pretty wild. Qwen 3.6 35b a3b much higher than 27b, cmon, bruv) deepseek low, right.
Qwen 3 Coder is doing great with standard coding and debugging, but like you mentioned, it scored pretty low on agentic tasks. Nothing wrong with that, it’s exactly what the benchmark is designed to show. The Qwen 3 A35B is a really solid model, you can’t argue with that. The coding and debugging are still more powerful for 27B models.
I’ve never been a big fan of DeepSeek v4, to be honest. I actually double-checked its outputs because of all the hype, but I honestly couldn’t find anything impressive.
Also, keep in mind that this benchmark just tests a few narrow categories and averages the results.
If you want the absolute best benchmark for frontier agentic coding, DeepSWE is the way to go. The only catch? Every model I can run locally currently scores 0% on it.
No problem. But for my stack either 35b a3b or qwen 3 coder are totally hopeless. 27b is kinda half way there but I guess its better at frond end, javascript but data science and backend with complex arch in C# dotnet + Python across multinode - not that much. 122b is almost there but I was switching to cloud every day. DS4F is only model I can work for days without calling the big brother like ds4 pro on cloud. Just the way it is. Sure it botches calls sometimes but fixes and gets there. That is what is important. Tests just scratch the surface - they never show end results, just the first attempt. Not a criticism to you, to all tests. I did build my own small custom bench where models were answering different questions, solving puzzles and actually coding via OpenCode CLI but I kinda showed it after switching to tool eval bench and settling with a model that works for me…
No hard feelings, mate! We just have to remember that the perfect benchmark doesn’t exist. Like you said, sometimes it really just comes down to that feeling of which model clicks best with your workflow.
With Toolery, all the models I’ve tested so far are giving me results that line up pretty closely with what you see on:
I’m not saying it perfectly mirrors their results, but it’s very close. And let’s not forget, all the major providers use that exact website to brag about how good their models are.
For me, the real advantage of Toolery is being able to test different quantizations at home to see exactly what performance we lose at each level. I don’t treat it as the absolute source of truth, it’s just another tool in the belt! :)
This image shared is probably close to how I see it, but minimax 2.7 is very limited on context, no go for my workload, I am close to or above 500k context every day. Mimo 2.5 - can’t test on 2 sparks but it would be tempting, however forum reports looping. Then we have ds4f and 122b and that’s exactly my lineup. 27b is capable, but slower, suffers from same speed degradation when yarn kicks in as 122b, and it certainly has less world knowledge than 122b by sheer number of weights, feels being trained for front end work, shallower than 122b on rest.
