# Tauergon agent harness optimized for local llm

**URL:** <https://forums.developer.nvidia.com/t/tauergon-agent-harness-optimized-for-local-llm/374265>\
**Category:** DGX Spark / GB10 Projects\
**Tags:** spark, nemotron, agentic-ai, llama\
**Created:** [June 23, 2026, 5:22pm UTC](https://forums.developer.nvidia.com/t/tauergon-agent-harness-optimized-for-local-llm/374265 "2026-06-23T17:22:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![alangeb](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@alangeb](https://forums.developer.nvidia.com/u/alangeb)\
**Post date:** [June 23, 2026, 5:22pm UTC](https://forums.developer.nvidia.com/t/tauergon-agent-harness-optimized-for-local-llm/374265/1 "2026-06-23T17:22:06Z")

</div>

Hi,

I’ve been experimenting creating my own agent harness purely to better understand - there are a lot of good (open source) harnesses out there, but for me doing myself is often the way to truely understand.

Eventually it turned into something I’ve started using daily, so I figured I could share as well (and named it tauergon in short).

> **[GitHub - alangeb/tau: Tau Coding Agent Harness](https://github.com/alangeb/tau)**
>
> Tau Coding Agent Harness

I’m truely just releasing this as concept - I think there are a couple of nice ideas - but not a polished nice UI system.

To ensure its working I’ve been running SWE Verified, looks like I have 67% with a Qwen 3.6 27b NVFP4 (although I will not be able to officially submit/get this verified based on new submission rules - not academia). (Update: running official validator locally shows almost identical pass rate)

> **[GitHub - alangeb/tau\_swe\_verified: SWE Verified benchmark - will not be able to...](https://github.com/alangeb/tau_swe_verified)**
>
> SWE Verified benchmark - will not be able to submit this for official as I'm not academia

Key ideas:

- tau agent fully capable of modifying itself - you want it changed, use it to change (and test) itself
- full agentic loop with tools
- tried to heavily optimiye for local AI:
  - maximizing prompt/prefix cache re-use to get fast loop times
  - multiple levels of llm response postprocessing, fixing, validating, rejecting (i.e. extracting toolcalls from reason/content, etc.)
  - fork tool to spawn (synchronous) subagent utilizing parents kv cache

- dream loop for continous self improvement
- pure python, zero dependancies (0!) - all you need is python (I tested 3.12.3 and 3.13.5)
  - will run very easy on almost anything recent
  - you can “see” (and change) the full end to end in code without diving into complex modules

Warning:

- This is dangerous and can execute arbitrary code (i.e. bash) - if you give it passwordless sudo even that - you should only use this experimentally in a sandboxed environment

Requirements:

- OpenAI compatible endpoint
- Python (3.12.3 and 3.13.5 tested)

I’m currently using this as a daily driver for a lot of my stuff:

- coding (some other proejcts I’m working on like a worldbuild, some local training, etc. …)
- configuring my systems, testing new models (and ways to serve them)
- playing with ways todo tools different (different file edit approaches, web search,

Notes:

- I am using a local searxng and crawl4ai to improve results - but it will work without them
- As you see in my hardware spec below I’m typically going for larger contexts (200k) - its not (yet) super context optimizes - but maybe something you’d like to ask it todo to itself ;-)
- I am currently running SWE Verified with it (and qwen 3.6 27b for now) - time consuming, but will publish as I have it
- I’ve been running this with different model classes (nemotron, qwen, gemma) mainly with llama.cpp and vllm (briefly tensorrt and sglang), this is currently my main driver (but that changes weekly):

Have fun! Experiment! What its really nice for is to ask it to modify itself…

**PC (to get interactive work done quickly - mostly single session):**

- Debian 13

- Nvidia RTX 5090

- cuda\_13.3.0\_610.43.02\_linux.run

- model is sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP, llama-benchy rates it around 10k/sec pp and 150+/sec decode

- docker-compose.yaml (depending on you system you might need to increase health check timeout for startup):

**DGX Spark (to drive multiple parallel batch work sessions):**

- sparkrun with @official/qwen3.6-27b-fp8-mtp-vllm

Alternative for NVFP4:

- cuda 13.0
- model is sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP - I am using this here to stay close to my other systems config
- startup script (I realize I should probably not use v0.23.0… but it seems to work):

```bash
#!/bin/bash
# Start Qwen3.6-27B NVFP4-MTP via vLLM Docker (froggeric chat template)
# - NVFP4 quantization with MTP speculative decoding
# - Uses froggeric/Qwen-Fixed-Chat-Templates (v19) for improved agentic loop & KV cache

set -e

CONTAINER_NAME="qwen36-27b-nvfp4-mtp"
MODEL="sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP"
IMAGE="vllm/vllm-openai:v0.23.0"
PORT=8000
HUGGINGFACE_CACHE="/home/alangeb/.cache/huggingface"
CHAT_TEMPLATE_URL="https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/resolve/main/chat_template.jinja"
CHAT_TEMPLATE_LOCAL="/tmp/chat_template.jinja"

# Download chat template if not present or outdated
if [! -f "${CHAT_TEMPLATE_LOCAL}"]; then
  echo "Downloading froggeric chat template..."
  curl -sL "${CHAT_TEMPLATE_URL}" -o "${CHAT_TEMPLATE_LOCAL}"
fi

# Stop existing container if running
if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
  echo "Stopping existing container ${CONTAINER_NAME}..."
  docker stop "${CONTAINER_NAME}" 2>/dev/null || true
  docker rm "${CONTAINER_NAME}" 2>/dev/null || true
fi

echo "Starting ${MODEL} on port ${PORT}..."

docker run \
  --rm \
  --name "${CONTAINER_NAME}" \
  --gpus all \
  -p ${PORT}:${PORT} \
  -v ${HUGGINGFACE_CACHE}:/root/.cache/huggingface \
  -v ${CHAT_TEMPLATE_LOCAL}:${CHAT_TEMPLATE_LOCAL} \
  -e VLLM_USE_FLASHINFER_SAMPLER=1 \
  -e VLLM_USE_DEEP_GEMM=1 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  ${IMAGE} \
  ${MODEL} \
  --host 0.0.0.0 \
  --port ${PORT} \
  --served-model-name model \
  --quantization modelopt \
  --linear-backend auto \
  --trust-remote-code \
  --language-model-only \
  --max-model-len 200000 \
  --gpu-memory-utilization 0.75 \
  --max-num-seqs 2 \
  --max-num-batched-tokens 2048 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --kv-cache-dtype fp8 \
  --calculate-kv-scales \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --chat-template ${CHAT_TEMPLATE_LOCAL} \
  --enable-auto-tool-choice \
  --enable-prompt-tokens-details \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  -tp 1 \
  -pp 1

```