Spark-DSV41 is a self-contained distributed inference stack and compiled Go Control Plane for serving DeepSeek-V4.1-Flash-1017 (EXL3 + full 203 GB Engram tables) across a 2-node NVIDIA DGX Spark (GB10, sm_121a) cluster over Dual-Rail ConnectX-7 RoCE v2.
It builds on the community TensorFold engines (MiaAI-Lab Native Zig and Elevatormusic Python/CUDA) and adds GB10 unified-memory protection, a targeted custom-hq hybrid quantization upgrade, and an agentic Go gateway (:8888) so the model works reliably as a daily driver for coding agents like Claude Code, OpenCode, and Cursor.
Why use Spark-DSV41 over stock recipes?
-
Zero UMA Page-Cache OOMs (
spark-dsv41-cacheguard)
ServingDeepSeek-V4.1-Flash-1017+ 203 GB Engram tables + Rank Caches touches ~576 GiB of disk-backed files across both Sparks. On GB10’s unified memory architecture, Linux page cache gradually eats intoMemAvailableuntilmemguardkills the engine.Spark-DSV41runs a lightweight C/Go daemon that issuesPOSIX_FADV_DONTNEEDevery 15 seconds on both nodes, keepingbuff/cachepinned at ~3 GiB for indefinite uptime. -
custom-hqHybrid EXL3 RankCache (Higher Coding & Tool Accuracy, Zero Thinking Loops)
Stock2.9 bpwcheckpoints occasionally get stuck in circular<think>self-doubt loops on hard algorithmic tasks.Spark-DSV41includes a one-command GPU RankCache upgrader (./scripts/build_custom_hq.sh) that streams ~12 GiB of unquantized source tensors via HTTP Range requests and re-encodes the high-sensitivity layers locally in ~31 minutes:- 8.0 bpw: Attention / MLA, Indexer, MTP (88.5% speculative draft acceptance), and LM Head
- 4.0 bpw: Shared Experts (unlocking
grouped_mma3_kernelon all 40 layers for 2,416 tok/s 8K prefill) - 3.0 bpw: Layers 18–22 hot routed experts
- Result: Eliminates circular
<think>timeouts (40–51% fewer thinking tokens on hard problems) and boosts LiveCodeBench v6 to 93.3% (28/30) (100%Easy,100%Medium,80%Hard).
-
Full Agent & Claude Code Compatibility (
spark-dsv41-gateway)- Native Anthropic
/v1/messages&/v1/messages/count_tokensendpoints with pre-configuredclaude-opus-4-6/claude-sonnet-4-6aliases. - Automatic
response_format(json_schema/json_object) translation for the Zig backend (raising structured output accuracy from33%to83%without sacrificing Zig speed). - Automatic DSML
<|DSML|function_calls>parsing, XML repair, JSON Schema scalar type coercion, and LoopGuard. - Maps
"max"effort requests cleanly to TensorFold’s"xhigh"tier instead of returningHTTP 400.
- Native Anthropic
Performance Snapshot (2x DGX Spark, Dual-Rail ConnectX-7 RoCE v2)
- Cold Boot Time: 26.0 s (
7.9sRankCache load +3003 MHzclock lock) - Single-Stream Decode: 61.3 tok/s (prose) / 111.8 tok/s (code with MTP)
- 4-Stream Concurrency (
llama-benchytg256): 96.4 tok/s avg / 112.0 tok/s peak - 8K Prompt Prefill: 2,416 tok/s
tool-eval-bench(Hardmode, Seed 42):- 88-Scenario: 95 / 100 (
168 / 176) - 97-Scenario (
v2.8.1): 93 / 100 (180 / 194, ★★★★★ Excellent) with a 2.8s median turn latency (~1.6x–2.5x faster wall-clock time thanGLM-5.3-FlashNVFP4)
- 88-Scenario: 95 / 100 (
- Coding Benchmarks (
temp=0.0,c=4):- EvalPlus HumanEval / HumanEval+: 96.3% / 90.9%–91.5% (completes all 164 tasks in 243–264s)
- LiveCodeBench v6: 93.3% (
28 / 30)
One-Command Quickstart
On a 2x DGX Spark setup with passwordless SSH between Head and Worker:
git clone [https://github.com/mazar/spark-dsv41.git](https://github.com/mazar/spark-dsv41.git)
cd spark-dsv41
./scripts/setup_from_scratch.sh
This handles RoCE v2 (MTU 4200) & clock tuning, builds the patched Zig container across both nodes, downloads the base checkpoint + 203 GB Engram shards, builds the custom-hq RankCache on your GPUs, and launches the engine + CacheGuard + Go Gateway on port 8888.
To connect Claude Code directly to your cluster:
ANTHROPIC_BASE_URL=[http://127.0.0.1:8888](http://127.0.0.1:8888) \
ANTHROPIC_API_KEY=spark-local \
claude --model DeepSeek-V4.1-Flash-1017
Huge thanks to MiaAI-Lab, bertholomus (TensorFold), Elevatormusic, SeraphimSerapis (tool-eval-bench), and eugr (llama-benchy) for the foundational engines and benchmarks!