Anyone tried mistralai/Leanstral-2603 · Hugging Face ?
It seems to be one model of the new Mistral 4 family of models.
Seems to be a perfect fit for DGX spark. Hope vLLM will run it soon. @eugr
4 Likes
It just has landed. 😂
First MoE of Mistral after a long time. They also mention a new family of models. So maybe they will come in different sizes.
Wondering which quantization tool will support the new architecture. There is no config.json to check.
Unquantized it needs two Sparks and a custom vLLM version for now.
I will try them this week on the spark :)
I know. Life is short ;-)
I’ll try it!
1 Like
They announced just now a Nemotron coalition while presenting Nemo Claw. Mistral AI is also part of it. And Black Forest Labs. Looking forward to that.
1 Like
There we go again!! Lets give it a try!!!
and another MoE - 119B parameters, with 6.5B activated per token.
Nice.
1 Like
Tja.
cosinus@vroomfondel:~$ docker pull mistralllm/vllm-ms4:latest
latest: Pulling from mistralllm/vllm-ms4
no matching manifest for linux/arm64/v8 in the manifest list entries
Build your own… :-D
@eugr we will need to be able to run it with vllm :)
Tried it on Nvidia NIM API, seems a little better than Qwen 3.5 122B on powershell based test, noticeably faster too, but not as good as OSS 120B (at least at powershell).
Yeah… personally, I found the Saudi and Korean models to be better, though it’s only been 1 day.