Agent Harnesses That Run Really Good Local AI for GB10 Systems

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Hey everybody, I just wanted to start a discussion around what agent harnesses people are using for local AI setups right now.

Personally, the two I keep coming back to are Hermes Agent and Agent Zero.

This community talks a lot about Hermes Agent, and honestly for good reason. It’s fantastic for local AI workflows. Mine runs through Minimax 2.7 and does an excellent job. It’s still more of a terminal based agent than anything else but it’s incredibly solid.

What I’ve found interesting though is that if you want more of a companion style local agent experience I find Agent Zero is my favorite, similar to the newer OpenAI Codex style apps or even the newer versions of Google Antigravity, there honestly are not that many projects that have been around long enough to fully mature and work reliably day to day.

That’s actually why my current daily driver has become Agent Zero. The community behind it has been updating it like crazy lately, and it keeps getting better.

One thing that’s especially interesting for the GB10 systems is how Agent Zero handles isolation and safety. It runs inside its own container and also uses its own virtual machine setup, which makes it feel a lot safer to experiment with while still keeping everything local.

image
image1157×571 125 KB

Some of the community members have now started actually installing Blender inside the container and then Agent Zero is the full access to it inside it, which is just amazing.

image
image1920×1031 208 KB

Like in Codex, you can actually select objects and then it can change it. And then it also has the full Libra Office Suite installed too, which is really good for report writing that I’ve been playing with.

image
image938×567 99.9 KB

Hermes Agent: https://github.com/NousResearch/hermes-agent

Agent Zero: https://github.com/agent0ai/agent-zero

Just wanted to bring this up and show the community some other agent harnesses that are working really well locally right now.

What are you all using these days?

2

Started with ClawdBot back in January. Rage uninstalled/reinstalled that thing over a dozen times, through all the name changes. Frustrating experience, but highly educational. Switched to Hermes Agent in early March, haven’t looked back.

Hermes, especially since the introduction of profiles, has been solid for me. I have tasks separated by domain, with dedicated profiles for each. But, I was burning API credits like they were going out of style. Which is how I ended up with a GX10 18 days ago. Took two days to get it working (most of that was re-wiring my brain to accept ‘things don’t actually always work the way you think they should’). Since 5 MAY, my main Hermes profile (assisstant/consigliere) estimates we’ve avoided approximately $500 in API credits, based on token burn and how much/frequently I was topping up various cloud providers’ portals.

A couple weeks ago I made the move from cloud Honcho to self-hosted. It’s not perfect, part of it is horribly broken (embeddings, specifically; meaning no peer cards or conclusions get saved). But, it works well enough. I could use my remaining OpenRouter credits for the openai embedding model, but the whole point of local inference (and self-hosted Honcho supporting Hermes) is data sovereignty, so I’m opposed to doing so.

I am aware of Agent Zero and even Space Agent. Interesting products. Their CEO hits the “tokenomics” aspect hard, but with local inference, I’m less concerned about that than I was a month ago. Will I try them (or other newcomers) out? Maybe. Am I already productive with Hermes and satisfied with the workflow? Absolutely. And, well, I’m used to it. I’ve learned how to get it (and each profile) to do what I need them to do.

tl;dr: my journey was chatgpt > grok > clawdbot/openclaw > hermes > local inference. Current configuration works really well for me. Might even add a second GB10-based system simply because the local inference experience has sent me down a LoRA-training rabbit hole the last two days. Local inference (and data sovereignty) is addicting.

3

NanoClaw has been my choice because I get to basically build it how I want it. Has the baseline work with the native Claude API integration using it locally with my Mac Studio and 4 GX10s with a combo of Qwen3.6 27B and Gemma 4 26B heretic builds on the Mac Studio and Llama 4 Scout and Mistral Small 4 and Mamba 3 SSM on the GX10s with a bunch of misc. utility models floating around Mamba acts as the arbiter in conjunction with OptiLLM

4

I think Hermes Agent, Pi Code agent, etc are good for experimenting usage case but I was not able to ensure it excute tasks with exact process so I mainly use them to test different usage scenarios.

When I want to excute specific workflow, I build it using

This way, I know exactly how agent will execute specific task.

5

same combo here agent zero and hermes . combining the qwen 3.6 35b dflash with the zai coding plan . works wonder and can’t get enough of it . created a skill that knows how to talk to agent zero via a2a .. now working on a self improving harness. wild days

6

First off, Thanks! for starting this thread, the discussions, with helpful info & details.

For me, the main driver for getting a small cluster of Asus GX-10, has been to run local inference, learn and also work (very little time spent) on side projects. Every weekend starts off the same - trying out new models, using the awesome community developed tools, figuring out what the LLM of the week is going to be.

Have Hermes setup and I had been mucking around with Claude CoWork (3P - 3rd party inference). Ready to give up , and moving to Agent Zero. Cowork 3P is in a constant state of changes, few weeks back, they required the model name to contain “Claude” & an Auth Key/Token. So I setup LiteLLM proxy, and a Cloudflare tunnel. The experience is pretty subpar. I spend more time trying to make it work. Now Agent Zero, I think it fits the bill.

TLDR;

Hermes works and I do like it but I haven’t found a replacement for Codex Desktop/ Claude Cowork. Agent Zero does seem to fit the bill.

I don’t want to drift this thread sideways from the main ask but any tips/tricks with Agent Zero ?must have skills? Just got it setup. Thanks!

Dan

Claude Cowork 3P - timeouts with Cloudflare, lack of local config

image
image2195×1174 281 KB

7

Beside the agents, I think this repo also worth a look :

Its aim is making all software agent native so you can access those softwares from your agent

8

How would you guys compare Agent Zero with Cline extension for VS Code?
Maybe is apples to oranges, but since I read here comparisons with Claude Cowork and Codex it’s something that may make sense.
I’m using mostly the extensions for vscode for coding tasks, just because they are more user-friendly once you need to be able to take a look at the code. But also they allow a single window with several different tools:

  • Codex
  • Claude Code
  • Cline with local agent

Cline is the closes I found that provides a similar experience to frontier harnesses in this context.

9

I’d say Agent Zero vs Cline is a bit apples to oranges, even though they overlap in some workflows.

Cline is primarily an IDE extension experience, similar to Cursor or the Codex extension in VS Code. It’s very convenient when you want the AI tightly integrated with your editor and codebase.

Agent Zero feels closer to something like Claude Cowork/Codex app or a standalone AI dev environment. It runs in its own isolated containerized Linux environment and exposes a web UI you can host locally or access remotely over SSH.

What I really like about Agent Zero on the DGX Spark is that I can host it once and access the same environment from my desktop, laptop, or even my phone. It becomes more like a persistent remote AI workstation than just an editor plugin.

If you still want local IDE workflows, you can install the Agent Zero CLI on another machine and connect directly back to the hosted instance. It also now supports computer-use capabilities, which pushes it further into autonomous agent territory.

So to me:

  • Cline = IDE-native coding assistant

  • Agent Zero = hostable remote AI dev environment with optional IDE integration

Both are great, but they solve slightly different problems.

10

You should try their CLI extension so it can work directly on your desktop and try computer use. They’ve got some pretty cool plugins too. I write a lot of documents for work, so I’ve been playing with the Libre Office integration.

11

Personally I use

IDE-native coding assistance
Antigravity, Visual Studio Code and Visual Studio Code Insider
all side by side - so I can work on multiple projects at the same time.
Or if someone has a hint how to open multiple instances of Visual Studio Code Insider that would also help.

Agents
With agents I do not have much experience yet.
I use OpenClaw in Virtual Machines on a proxmox Server and Opencode.

Have not found the intuitive easy to use gui / cli tool yet to manage and orchestrate agents and projects.

12

Ctrl + Shift + N → New window

13

Great setup! I use the mlx-optiq.com quants for my mac with qwen 3.5 27b to run hermes

14

Guys, is there a particular reason you want to use standalone agent software like hermes or zero to actually code with? Whats wrong with OpenCode or Pi? I have two Hermes setups on my sparks, one for different project/company, with karpathy wiki, sould, etc. While its good I can’t get off the idea I would love to combine OpenCode with Hermes, as OpenCode is just sooo freaking good for interactive work, while Hermes is good for standalone. I can use OpenCode to talk to Hermes (it has open-ai endpoint) but its just for chat, ideally I would want hermes agent to use opencode for its amazing MCP and modularity. So I am currently have Hermes Agent to use OpenCode cli to do tasks, instead of just bashing it. It’s hilarious :)

15

Good question. For me, having dedicated Hermes profiles to separate flows just works. The output quality has been acceptable. That being said, I’ve recently started looking in to OpenHands for the coding tasks with Hermes Agent orchestrating it. Not sure if it will degrade, improve or not change things. But, with AI doing the grunt work, I suddenly have cycles to burn on experimenting with potential optimizations.

16

As for coding tools, I used to use OpenCode, but I didn’t find its approach and logic effective enough. Now I only use Kilo Code + Visual Studio Code.

17

For me, nothing beats a well-tuned multi-model setup (4x GB10s with minimax 2.7, qwen3.6 35b, and now gemma 4 12b with mimo 2.5 pro API as the cheap-as-s*&t orchestrator and brain) in Hermes with the ability to call up Codex or Claude Code for edge cases… I think I have tried at least 12 harnesses and I keep just coming back to good old Hermes.

18

For most coding or sysadmin tasks, I like opencode. I have to force it to search the web more often than it is accustomed to, and it has a bad memory system stock, but it works very well for most of the tasks I need an LLM for.

I have been trying to like Hermes for about a month now, but using the same model (MiniMax 2.7 on 2x Sparks) it just feels slow and…dumb. I have even installed clawmem to help with learning over time and I just cannot get on with it right now.

Agent Zero is on the homelab road map, just need to find time.

19

I have also been trying Hermes on my MacBook Air for the last month with my DGX Spark serving (via vllm using eugr’s docker) Qwen 3.6 27B and 35B. I have tried various quants (NVFP4 all the way up to bf16 on 35B) but settled on rdtand’s PrismaQuant.

I did not yet do much config changes on the Hermes side yet but it quite often stops mid task (in a thinking step), have any of you encountered this? And if so, how did you fix it?

20

I’ve tested OpenCode, omp, CodeWhale custom build for local models, CC, Hermes. And for now I can say that combo like Hermes + Qwen3.6-27B (PrismaQuant) works the best in terms of delivery. It feels slower than 122B or 35B models from Qwen, but quality is my main aim. For additional tools for harness I use Sentrux for giving architecture constrains and Codegraph for codebase analysis, Engram as persistent memory.