Those who use their spark for agentic development workflows what is your setup? I have tried all the inference engines and all the models and keep switching hoping to find one that stands out. I think right now Flash Next is where it is at, does anyone else have a different opinion?
I have tried the Azampatti INT4 that has the reduced experts and that has given me the most consistent speed, Mia Tensorfold is pretty cool too but had some tool calling issues.
I’m currently testing Athena Inference Engine which is awesome for wanting to fast switch between flash next and ds4 flash but I find the speeds aren’t what I was hoping for.
I think Qwen3.8 Flash Next is the best model for a single spark as long as PLE can be offloaded to the NVMe. Qwen3.8 27B will be quite a bit slower. Take a look at the flash next FP4 checkpoint recently released from NVIDIA. Based on my understanding it is still NVFP4 but a bit more aggressively quantized such that it run decently on a single spark. QAD is applied to compensate the quality loss in quantization.
Yes I’m currently comparing the FP4 release to the NVFP4 with PLE offloaded and also trying to compare it to @azampatti INT4 version that has less active experts.
Do yourself a favor and just run this Cockpit LLM manager. Once installed, it gives a web dashboard to install the most optimized versions of LLM.
I’m running qwen-3.8 flash - nvidia release.
Running it for days on coding large project. Averaging 32 tok/s. Local llm now has the ease of frontier , with token speed that make it worth running local.
