Seeing a lot of buzz and questions about Agents A1. I dont have hardware to set up testing, only a single DGX spark currently. Only ran into 1 post that said it was benchmaxxed.
If anybody has input on any of these models, (or even how i could provide my own testing results. I am using these models as API endpoints for my team to use for compute, so I prefer vLLM. Darwin was created for for llama.cpp only
I posted a brief write up on Agents-A1 as an X article a few days back. I ran it for a while, seemed competent. Agree with @jc2375 that it’s a Qwen finetune. It benchmarked (llama-benchy, tool-eval-bench, spark-bench) better/higher than Qwen3.6-35B-A3B on a single GX10 node. Didn’t notice any degradation going to NVFP4; no tool call fails, looping or hallucinating. Seemed pretty solid.
I’ve since moved on to playing with Gemma4-26B-A4B-NVFP4 for a couple of Hermes profiles. Seems crisper, just as capable. Also benchmarked slightly higher than Agents-A1. So far, it’s just as solid as Qwen3.6-35B has been for my use cases.
Edit to add: I’ve been running these models in their various quants via spark-vllm-docker. No problems at all. Well, other than Gemma4 has an issue with vLLM and vision, but there’s a fix for that.
I’ve never had an under-40B model actually do well with agents. If for some reason my sparks crap out and it falls back to Qwen 27b, or 35B, it may survive a turn, but once it has to install software on another machine within the network, or understand a bleeding edge repository and apply to local hardware constraints, it isd completely useless. Qwen-397b, Deepseek V4 Flash have been the best so far to the point that I only look for 100+ billion parameters for agentic backbone.
27b is a gem of tool calling, really. I tested a very good model once, largely forgotten already - iQuest IQcoder 40b dense (double-pass, non-thinking) it was very solid but very slow too - 40b dense and no mtp. But only 128k context, so out of most use cases already.
You will not regret it, I felt it was the first real local usable AI model on the sparks—fast for agents, as good as Sonnet 4.5 by all accounts, no quantizations needed—original weights released by deepseek. Runs for weeks on the cluster nonstop.