Benchmarks look fantastic, should be a new king for single spark.
all the quantifications so that we waste time on other quants
Those actually aren’t the other quants. Poolside provided DFlash drafters specific to each of their official quant targets!
Haven’t seen that from anyone else before. It’s a great quality of life choice to get us going ASAP - fast.
Looks like, tested it with thinking of and results where amazing. Managed even to get 40-50 tokens/s at c=1 and 80-100 tokens/s at c=2.
This is probably important, tried first model with reasoning and it was completely useless. Turned it off, and started to get results.
Benchmarks looks great indeed. But lack of vision support is a big downer :-/
going to make a prismaquant-gridbook release of this soon!
waiting for your prismaquant version!
it does not perform that well, its not new king.
I got 22.6 t/s on this. Not sure what the author was smoking.
On rtx6kpro discord w/them working out kinks w/the model KLD on quants and probable need to re-create the dflash models. Expect this to stabilize as with all models over the next couple of weeks.
Never seen a team this engaged w/a community and rapidly working through improving things so quite optimistic about this.
Just a heads up, Poolside has updated the NVFP4 weights and .jsons. Just now pulling them, so no idea what changes.
i loaded it and tried a bunch of different builds, not that impressed, but it is small
Pulled latest and ran my agentic workflow on it. Definite improvement but still not usable for me.
On a simple bug fix that took q3.6 27b about 10 minutes to fix, Laguna spent 15 minutes deciding what to do, implemented one file, then stopped midsentence randomly. I woke it up, then it turned around and hallucinated a path, and stopped randomly again.
At this point I’m thinking the nvfp4 quant is just not usable right now.
I do not smoke. Boy, watch your language.
I made this post because this model could potentially be the successor to Qwen 3.5 122B. The big question is: Are the benchmarks real? Did we really get a model that is better than Qwen? This is quite important for single- or even dual-Spark users because it is the perfect size to fit alongside the KV cache.
It has only 8B active parameters, so it could potentially reach 45-50 tokens per second with AutoRound. However, I do not think speed is a concern for most people anymore. We have reached a decent level of understanding of how much performance we can squeeze out of the Spark.
Ran it on a single Spark. Benchmarks did not translate for me.
Official NVFP4, 256K context. 19-24 tok/s decode, TTFR ~0.36 s.
Tool evals, same suite (Seraphimserapis) against Qwen3.6-35B-A3B, both with thinking.
Short: Laguna 97/100, Qwen 100/100
Hardmode: Laguna 86/100, Qwen 91/100
Laguna had more issues with failures on tool forcing, polling, recovery and rollback
Also it’s really bad in german (I know, not important to many but to the German community here for sure)
So no new king there for me at all… Pretty disappointed.
Hot off the silicon
their new updated NVFP4 Quant seems to have unlocked some performance especially in the coding side
45.5 t/s code / 27.2 structured
Follow-up: reran the temp sweep on the updated NVFP4 (07614121), same config as my earlier post. The curve flattened:
| temp | old accept / decode | new accept / decode |
|---|---|---|
| 0.0 | 51.9% / 44.2 | 41.3% / 40.7 |
| 0.2 | 39.9% / 40.9 | 43.3% / 42.3 |
| 0.7 | 28.8% / 32.2 | 34.6% / 38.7 |
So the new quant gives back some temp-0 but picks up a lot at sampled temps (+20% decode at 0.7).
I still cannot get anywhere near the numbers reported in GitHub - MiaAI-Lab/Laguna-S-2.1-DGX-Spark-RTX-6000-PRO: vLLM 0.25.1 serving stack for poolside/Laguna-S-2.1-NVFP4 with DFlash speculative decoding — DGX Spark & RTX 6000 PRO · GitHub as I mentioned previously.