Hey guys,
Check out this new release: Nex-N2-Pro, a 397B parameter MoE model based on Qwen.
https://huggingface.co/nex-agi/Nex-N2-Pro
From what I’m seeing, it looks like it can run on just 2 Spark nodes. Definitely an interesting option for anyone experimenting with large-scale reasoning models without needing a huge cluster.
Has anyone tested it yet? Curious how it compares to other recent Qwen-based models.
Interesting! Which quantization are you thinking to squeeze this into 2 Sparks?
Well, we are in the same territory of Qwen3.5-397B-A17B, in both active and total number of params.
FP8 quants would fit a two Spark cluster, albeit barely. In order to squeeze more context, NVFP4 or Int4 quants might be needed; but something more optimized based on PrismaSCOUT would be definitely nice.
What strikes me most is that with its 397B params New-N2-Pro fares better than DeepSeek V4 Pro (1.6T params) on most benchmarks.
Being able to fit this model on just two nodes setup is turning point, because it significantly reduce the need for proprietary API-gated solutions, as well as those open-weight models (a la DeepSeek V4 Pro) that are still demanding to run in home labs.
Didn’t flesh out to be as great as the benches suggest: https://www.youtube.com/watch?v=HDaUWU-3NyE
I gave this a try, on tool-eval-bench it scored 80, and I get around 90 with qwen 397b, so they may have made some improvements, but I’m not seeing it so far.
Maybe with more refinement it would be better? since there is no real way of knowing what improvements were actually made, its really hard to judge against the original qwen, as all of these scores against other llm’s there is no way to actually know what is being run, so its hard to compare apples to oranges.
I see you point, and, indeed, there is a reproducibility crisis here. At the same time, the lower score may result from other moving parts too, such as inadequate chat templates (?). Just wondering.
On a related note, Nex-N2-Pro is now available on OpenRouter. I will be playing with it a bit before eventually giving it a try on prem.
A quick update: Having now spent some time with Nex-N2-Pro, I am more than glad to say that I am favourably impressed by it.
Far from wanting to replicate standardised test benches, I have run a number of prompts, in high thinking mode, which were previously run on GPT 5.5 GPT 5.4, Gemini 3.1 Pro, and DeepSeek V4, so as to have my gut feeling about its quality.
The overall performance level attained goes certainly beyond the parameters count would suggest. We are on par, and sometimes beyond, DeepSeek V4 Pro, at a much smaller memory footprint. On a few occasions the model reasoning entered an inconclusive loop; however re-running the same prompt a second time eventually led to a quality answer. The prose is more terse than what is generally observed on GPT and Gemini, but this can be fixed by more explicit prompting.
EDITED TO ADD: The prompts revolved around: peer-reviewing research papers, logic reasoning, probability calculus, brainstorming, and some creative writing.
Could you confirm which quantization you used? Additionally, would you be able to share your recipe?
I have downloaded the NVFP4 version, but I have not been able to get it running on two Sparks
As I briefly mentioned above in the thread, I am giving it a test ride on OpenRouter first, before playing with it on prem. There are currently two OpenRouter providers for this model, Nex AGI and SiliconFlow. While no quantization is specified on the UI, I expect they are running the native FP16 quants.
Edited to add: As this is Qwen3.5-397B derivative work, I would recommend to develop a specific recipe starting from Owen’s as a template.
Lol, anything I ever run on openrouter scored significantly lower that same model runningocally in Nvfp4, int4 autoround 9r fp4/fp8. Every time. I suspect 5gey run below fp4, maybe q3 or q2. Nobody every runs bf16, insanely expensive
Even better then: As the quality observed on OpenRouter is truly good for this model, when the opportunity will come to run it on prem there is the possibility it will really shine then.
Also, the incentives for the challengers (i.e., N2-Pro) should be aligned so as to promote greater inference effort (i.e., greater dynamic range in the quants), if they want to succeed at eroding market share.
FP8 is 1 byte per weight, so 397 GB, I don’t see how it could fit a two Spark cluster.
4 bits is the current used quant for Qwen 3.5 397B, and it fits only one full context.
Did you mean something else?
Yes, indeed we are in the same ballpark of Qwen3.5-397B, from which this model is derived from. NVFP4 or PrismaSCOUT with custom quants is the way to fit this on 2 units cluster.
Is this could fit 2x DGX Sparks I would be happy.
It should be, I never tried Qwen 395B but I heavily used 122b and it is still one of the best models 2xSparks can run. Tool Bench extra hard (79 tests in 2.0) scores it 91-92, while DeepSeek v4 Flash hits max at 89. I switched to DS4F for the 1M usable context window which I really need (I feed massive cache and then keep working with it hitting 90%+ on prefix cache) - 122B craws and cries after 256K with YaRN. So I would imagine 395B should be at least slightly better. And yes, brother, local is the only way!
Said that I am now prepping a key document with my agent to employ Fable 5 (Claude Mythos for public released overnight) to give us a critical review and recommendations of the key principles of our quant trading system. Lol. But even if I blow 50 bucks in one prompt it worth it.
PS did the Fable 5 review - it’s bloody brilliant. Best 6 dollars spent in 20 minutes ever. Based critique, sound recommendations, factual and thoughtful. Definitely will keep it as a weekly auditor/external consultant model. Knowledgeable about financial markets as quack!
PPS: used via open router, one pass on normal reasoning, final on x-high. Normal very fast, x-high slow, which is a good sign, he re-thought and appended a lot. Same pricing on OR as Anthropic direct: 10/50, Venice for example change 12/60. And both charge fees/taxes on deposit of course.
Look what I have found. Reaped so it fits 2xsparks, AFAIK same actives retained. Int4. 145gb weights, same as ds4f fits with lot of cache and room to breathe
Thanks for sharing. I am always cautions about REAP-ed models, because they are highly domain sensitive. So, they work fine with any given application workload, as long as the calibration dataset used during the pruning well support the said application domain; but they work less well elsewhere.
A more straightforward model quantisation, on the other hand, is expected to exhibit more consistent performance degradation, if any, and becomes more amenable to be used as all rounder.
I can’t speak for this particular model but IMO it’s great to reap unneeded weights if they don’t affect your intended use. And a quick test shows it. I am totally fine if they reap Shakespeare, French poetry and Mao’s teachings, lol :)
