Scaling a 22-Agent Dev Team (Qwen Orchestrator) on a 2x Gigabyte AI TOP Atom Cluster (PCIe 5.0) using sparkrun and Qwen Code

Hi everyone,

I wanted to share a multi-agent orchestration project I’ve been developing and stress-testing on accelerated compute. With the growing focus on local, high-performance agentic workflows, I built Qwen Orchestrator—an extension designed exclusively for the Qwen Code CLI that transforms a single terminal assistant into a coordinated team of 22 specialized agents.

⚙️ Hardware Setup & Test Cases (sparkrun)

To handle the heavy parallel inference required by multiple agents operating simultaneously, I deployed the orchestrator on a cluster of 2 Gigabyte AI TOP Atom nodes.

For the test cases, I utilized sparkrun to efficiently distribute the workload across the cluster while running the Qwen 3 Coder Next model. This distributed setup handles the heavy lifting perfectly, allowing the Commander agent to constantly stream context updates to background workers (like the Frontend and Backend agents) doing parallel code generation without bottlenecking.

🛡️ Solving the Long-Context Loop Problem

One of the biggest bottlenecks I encountered during these high-throughput tests was LLM degradation in massive context windows. When orchestrating 22 agents, the context gets huge fast, and I noticed the model would occasionally fall into random, infinite reasoning loops—wasting valuable GPU cycles.

To fix this, I implemented an “Anti-Loop” Monitor Agent. It acts as a continuous background watchdog that analyzes the reasoning stream. If it detects redundant logic loops or stalled execution in the long context, it automatically interrupts and redirects the agent, saving compute time and keeping the cluster utilization optimal.

🚀 Core Architecture Highlights

Beyond the hardware scaling, here is what the orchestrator actually does:

  • True Parallel Execution: The Planner breaks down tasks, and sparkrun helps distribute the concurrent generation from the Frontend, Backend, and QA agents.
  • Strict “No Lazy Code” Policy: I wrote an anti-pattern skill that outright forbids agents from outputting placeholders (e.g., // TODO: implement later). The Tech Lead agent forces complete CRUD generation.
  • Persistent Knowledge Graph: Uses an MCP (Model Context Protocol) memory server to store architectural decisions across sessions, reducing the need to re-feed context to the GPUs on every new prompt.
  • Zero Vendor Lock-in: It’s built entirely for the CLI and local models.

🔗 Links & Setup

If anyone here is experimenting with multi-agent workflows on their DGX or custom clusters, I’d love to hear your thoughts or see how it performs on your setups.

Any feedback on optimizing the prompt routing for multi-node setups is highly welcome!

This looks very cool! I did build a load balancing proxy in golang for speed - you can create separate virtual routes or you can create clusters which are context aware and pin your agent to a particular node where it’s context is cached - GitHub - wentbackward/hikyaku: A superfast proxy and smart load-balancer for AI Inference — virtualize models, share local and provider back-ends, optimal caching, lock-in sampling parameters, debug message flow and obtain OTel metrics · GitHub - whilst I have load and soak tested it and it can handle hundreds/thousands of concurrent connections on low end hardware, I’ve not tried it on anything like your scale - right now it’s still holding the full context in memory whilst a request is in progress - it will almost certain be memory constrained. LiteLLM seems to now have a proxying service in beta, otherwise, I could only find commercial solutions. Oh, I also recently added a trivial loop detector - it hasn’t had the same testing rigor as the code router.

Do you have a demo video?

No no no no no,

I bought a Gigabyte AI Top Atom for £3,599 and now they are £3,949 and I want a second one but I’ve talked myself off the edge.

Here you are pushing me back 😂