I have a question. For MoA, why did you use Hermes and have you got some good advices there besides the aggregator / reference split.
honestly on my machine Hermes did cause a lot of issues:
so I ended up uninstalling it and now I think of using OpenWebUI and apply MoA there.
What is your opinion?
These are good questions, and ones we had come across. We evaluated about a dozen or so different harnesses before landing on Hermes. It wasn’t the most feature-rich, but because it can learn over time, we thought if we invested the initial time, it would eventually learn how to run our workflows.
Turned out to be true (but certainly not without it’s faults). Hermes runs in several of our production environments, including the DGX Spark. Several things we had to do to mitigate some of the problems with Hermes:
Almost never update – Yes. we don’t need every new feature, we want a working production system. Once a system runs exactly the way we want it to, it’s pretty much locked into to that version, unless it’s a complete revamp (like on the DGX).
MoA – We run MoA on demand, thus only activating it if it’s needed in a conversation, and then we exit back to a single model. For us, that give us the best of both speed and intelligence. Since MoA activates both models as advisors, the judge (Flash-Next) takes in both responses in parallel.
Endless loops – we hit this exactly: a model + MTP speculative decoding combo can fall into a tool-call attractor (one of my agents burned 176 identical calls in 28 min; MTP’s high acceptance rate on boilerplate text locks the repetition in). It’s a model/server-level behavior, not the framework’s fault, meaning it would reproduce on any client. The fix is a client-side hard stop on repeated calls (Hermes has tool_loop_guardrails.hard_stop_enabled: true — we recently enabled to block at 5 identical calls instead of only warning).
Hallucinating – Models get overconfident, that is actually by design funny enough. Even Flash-Next will hallucinate sometimes because technically it’s only 6 billion active parameters at any given moment. You need something like the 27B dense or more, and then, the hallucinates drops off.
The other is the harness and how it’s managed. For examples, Hermes won’t come out of the gate working exactly as expected, but learns over time. While we cannot eliminate hallucinations entirely, with these steps, I have pretty much eliminated 100% of all hallucination problems.
Hermes Agent Persona
You are a professional, smart expert assistant, always willing to learn and better yourself. Be direct, accurate, and efficient in all responses. Prioritize facts, truth and clarity. Always think before you act, and respect boundaries of your scope and when to seek permission.
MIDMEMORY.md (~/.hermes/memories/) = overflow tier. DO NOT ERASE. If MEMORY.md full, move secondary items there (write_file — memory tool can't touch it). Read it (read_file) when info not in MEMORY.md. Not auto-loaded.
Hanging – with local llama.cpp backends, this is almost always slot exhaustion or context overflow: the member request errors/waits server-side while the orchestrator blocks forever. Per-member timeouts + treating a timed-out member as “no contribution” fixes 95% of it. In another words, having Hermes do too much while the model’s engine (llama.cpp) couldn’t keep up.
Hangs require more investigation into how Hermes is used. For example, for us we worked out 96K per context, but cut system prompt and tool definitions by 50% so that each call becomes faster/less tokens used per turn.
Model: Qwen3.8-Flash-Next
Window: 98,304 tokens
In use: 58,179 / 98,304 (59%)
██████████████░░░░░░░░░░
Headroom to limit: 40,125 tokens
Auto-compresses at: 73,728 (75%) — 15,549 to go
Compressions this session: 0
Session totals (cumulative across 1 API calls)
Input 58,179 · Output 717 · Reasoning 0
Total billed: 58,896
Throughput, not context size — each call re-sends the window above.
Estimated usage by category
■ System prompt 7,858 tokens 8.0%
▣ Tool definitions 11,049 tokens 11.2%
▦ Subagent definitions 938 tokens 1.0%
▧ Memory 895 tokens 0.9%
▨ Conversation 30,903 tokens 31.4%
· Free space 46,661 tokens 47.5%
Context window: 58,179 / 98,304 tokens (59%)
Use /context all for per-skill and per-toolset costs.
Hey thanks a lot, I appreciate the time you take to answer my questions and concerns.
Thanks already ahead for your time replying my questions here.
No worries. I actually have this forum to thank for helping me fit a few pieces together from different people’s solutions, so if documenting what I found (and my own findings) helps someone else, I think that’s all the better.
Hermes does not currently auto-classify a request and decide to enter MoA …but I think there is a way to design that behavior.
First, the way I use MoA today is explicit:
/model, where a Mixture of Agents preset can be selected much like another model/moa when I want it for a particular requestFor automatic MoA escalation, Hermes can delegate work to subagents, and MoA presets are exposed internally as selectable provider/model pairs.
That creates an interesting design:
Flash-Next handles normal work itself
Flash-Next → normal response
But when the main agent decides a task needs deeper review or an independent second perspective:
Flash-Next → delegate_task → MoA specialist → result back to Flash-Next
In other words, the MoA becomes a specialist that the main agent can call, rather than something that has to be active for every request.
I have not been running my production system this way yet, so I don’t want to present it as something I’ve validated. But the current Hermes delegation/provider architecture appears to support this composition, and I’m interested enough that I may test it.
On LLMs: my background is actually not that different from yours — ANNs, smaller models and more traditional statistical methods.
One thing I’ve learned with agentic systems is that I assume the harness is untrained until proven otherwise. I take what I’d call a “productive skepticism” approach.
In my experience, getting the system running is maybe 5% of the work. The other 95% is trial, observation, failure analysis and repeated testing.
Production readiness becomes a statistics problem:
You might decide, for example, that 98 out of 100 representative tasks completing acceptably is enough for one workflow — while another workflow involving accounting or contracts may need a much higher threshold and mandatory review.
I can definitely see why MoA could be useful for your use case, whether through Hermes or another orchestration layer. You may care much more about correctness and independent validation than latency.
I also suspect systems like this eventually benefit from supervisor/validator agents overseeing worker agents. That is broadly the direction we’ve taken in our own production systems.
On the UI question: I agree with you that Hermes itself is probably not what I would expose directly to a normal business user.
For end users, I would strongly consider a deterministic application interface — web app, forms, structured workflows …where they provide company data in a controlled way.
Chat can sit on top as the conversational layer, but I would consider if making free-form chat the only interface for accounting, contracts or other high-consequence workflows is the best option.
I’d separate the layers:
UI / web app → workflow/orchestration → model gateway → models/tools/data
That gives you much more control over permissions, validation, auditability and what actions the AI is actually allowed to take.