Spark-DSV41: Production-Hardened DeepSeek-V4.1-Flash-1017 Engine & Go Control Plane for 2x DGX Spark

GitHub: GitHub - mazar/spark-dsv41: High-Throughput DeepSeek-V4.1-Flash-1017 Engine & Go Control Plane for 2x NVIDIA DGX Spark (sm_121a) · GitHub

Spark-DSV41 is a self-contained distributed inference stack and compiled Go Control Plane for serving DeepSeek-V4.1-Flash-1017 (EXL3 + full 203 GB Engram tables) across a 2-node NVIDIA DGX Spark (GB10, sm_121a) cluster over Dual-Rail ConnectX-7 RoCE v2.

It builds on the community TensorFold engines (MiaAI-Lab Native Zig and Elevatormusic Python/CUDA) and adds GB10 unified-memory protection, a targeted custom-hq hybrid quantization upgrade, and an agentic Go gateway (:8888) so the model works reliably as a daily driver for coding agents like Claude Code, OpenCode, and Cursor.


Why use Spark-DSV41 over stock recipes?

  1. Zero UMA Page-Cache OOMs (spark-dsv41-cacheguard)
    Serving DeepSeek-V4.1-Flash-1017 + 203 GB Engram tables + Rank Caches touches ~576 GiB of disk-backed files across both Sparks. On GB10’s unified memory architecture, Linux page cache gradually eats into MemAvailable until memguard kills the engine. Spark-DSV41 runs a lightweight C/Go daemon that issues POSIX_FADV_DONTNEED every 15 seconds on both nodes, keeping buff/cache pinned at ~3 GiB for indefinite uptime.

  2. custom-hq Hybrid EXL3 RankCache (Higher Coding & Tool Accuracy, Zero Thinking Loops)
    Stock 2.9 bpw checkpoints occasionally get stuck in circular <think> self-doubt loops on hard algorithmic tasks. Spark-DSV41 includes a one-command GPU RankCache upgrader (./scripts/build_custom_hq.sh) that streams ~12 GiB of unquantized source tensors via HTTP Range requests and re-encodes the high-sensitivity layers locally in ~31 minutes:

    • 8.0 bpw: Attention / MLA, Indexer, MTP (88.5% speculative draft acceptance), and LM Head
    • 4.0 bpw: Shared Experts (unlocking grouped_mma3_kernel on all 40 layers for 2,416 tok/s 8K prefill)
    • 3.0 bpw: Layers 18–22 hot routed experts
    • Result: Eliminates circular <think> timeouts (40–51% fewer thinking tokens on hard problems) and boosts LiveCodeBench v6 to 93.3% (28/30) (100% Easy, 100% Medium, 80% Hard).
  3. Full Agent & Claude Code Compatibility (spark-dsv41-gateway)

    • Native Anthropic /v1/messages & /v1/messages/count_tokens endpoints with pre-configured claude-opus-4-6 / claude-sonnet-4-6 aliases.
    • Automatic response_format (json_schema / json_object) translation for the Zig backend (raising structured output accuracy from 33% to 83% without sacrificing Zig speed).
    • Automatic DSML <|DSML|function_calls> parsing, XML repair, JSON Schema scalar type coercion, and LoopGuard.
    • Maps "max" effort requests cleanly to TensorFold’s "xhigh" tier instead of returning HTTP 400.

Performance Snapshot (2x DGX Spark, Dual-Rail ConnectX-7 RoCE v2)

  • Cold Boot Time: 26.0 s (7.9s RankCache load + 3003 MHz clock lock)
  • Single-Stream Decode: 61.3 tok/s (prose) / 111.8 tok/s (code with MTP)
  • 4-Stream Concurrency (llama-benchy tg256): 96.4 tok/s avg / 112.0 tok/s peak
  • 8K Prompt Prefill: 2,416 tok/s
  • tool-eval-bench (Hardmode, Seed 42):
    • 88-Scenario: 95 / 100 (168 / 176)
    • 97-Scenario (v2.8.1): 93 / 100 (180 / 194, ★★★★★ Excellent) with a 2.8s median turn latency (~1.6x–2.5x faster wall-clock time than GLM-5.3-Flash NVFP4)
  • Coding Benchmarks (temp=0.0, c=4):
    • EvalPlus HumanEval / HumanEval+: 96.3% / 90.9%–91.5% (completes all 164 tasks in 243–264s)
    • LiveCodeBench v6: 93.3% (28 / 30)

One-Command Quickstart

On a 2x DGX Spark setup with passwordless SSH between Head and Worker:

git clone [https://github.com/mazar/spark-dsv41.git](https://github.com/mazar/spark-dsv41.git)
cd spark-dsv41
./scripts/setup_from_scratch.sh

This handles RoCE v2 (MTU 4200) & clock tuning, builds the patched Zig container across both nodes, downloads the base checkpoint + 203 GB Engram shards, builds the custom-hq RankCache on your GPUs, and launches the engine + CacheGuard + Go Gateway on port 8888.

To connect Claude Code directly to your cluster:

ANTHROPIC_BASE_URL=[http://127.0.0.1:8888](http://127.0.0.1:8888) \
ANTHROPIC_API_KEY=spark-local \
claude --model DeepSeek-V4.1-Flash-1017

Huge thanks to MiaAI-Lab, bertholomus (TensorFold), Elevatormusic, SeraphimSerapis (tool-eval-bench), and eugr (llama-benchy) for the foundational engines and benchmarks!

2 Likes