DeepSeek-V4-Flash (official FP8) running across 2x DGX Spark — TP=2, MTP, 200K ctx, recipe + numbers

tonyd615 · June 5, 2026, 4:38am

Idk if I need to make a GitHub or something but I got this thing running 8 concurrency with 500k context and it’s fast.

a.grmbl · June 5, 2026, 6:11am

Just got second Spark, was running 122B for some time on single node - DS4F feels much more powerful. And at 40 tps it’s almost the same speed I’m getting from Anthropic these days.

I run summarizing RAG stack using local model (processing raw session logs), so have real-world corpus to evaluate quality (tool benches do not tell the whole story - Qwens 3.6s are good at tools but dumb at general use).

After prompt tuning (telling Claude to rewrite them using DS guidelines) it is definitely better than Qwen 3.5 122B - less hallucinations, follows instructions well, extraction quality is clearly better.

In Claude Code and Agent0 it really feels almost like using Anthropic (especially on Anthropic’s bad days, when they enable INT0-level quants) - tool calls are to the point, synthesis is correct and concise, also the tone feels really good.

IMO, we’ve got new king here, well done, DeepSeek.

corbett_korbett · June 5, 2026, 10:15am

how did you get it working correctly? Mine outputs gibberish.

a.grmbl · June 5, 2026, 10:38am

Did not do anything exotic, just followed recipes in this and linked threads. Took half an hour to do with Claude Code.

Definitely no gibberish, agent just returned from 30-minute loop of market research, findings are valid and well-reasoned.

wolttam · June 5, 2026, 8:25pm

My PSA: if using DSv4 Flash for coding, it helps A LOT to use xhigh/max thinking effort

paxren2020 · June 5, 2026, 8:46pm

If you build the community image with @jasl PR, you get an 18% speedup on prefill compared to last week’s version without MTP. (By the way, I’m not sure how to accurately benchmark generation speed with MTP, as llama-bench is showing worse results with it enabled).

results

week ago

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d1000	918.11 ± 100.53		2029.42 ± 238.55	2028.05 ± 238.55	2029.48 ± 238.55
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d1000	25.33 ± 0.06	26.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d10000	1142.69 ± 50.89		8468.58 ± 502.26	8467.21 ± 502.26	8468.62 ± 502.26
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d10000	23.71 ± 1.20	25.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d15000	1203.59 ± 25.76		11922.26 ± 192.96	11920.89 ± 192.96	11922.31 ± 192.96
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d15000	23.61 ± 0.90	25.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d20000	1165.12 ± 43.71		16071.92 ± 657.14	16070.55 ± 657.14	16071.97 ± 657.15
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d20000	23.76 ± 0.27	24.33 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d30000	1123.12 ± 12.54		24705.52 ± 293.46	24704.15 ± 293.46	24705.76 ± 293.30
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d30000	23.29 ± 0.04	24.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d40000	1105.78 ± 2.36		33289.92 ± 228.65	33288.55 ± 228.65	33289.96 ± 228.65
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d40000	22.62 ± 0.07	23.33 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d70000	1003.92 ± 39.04		63426.17 ± 2618.50	63424.80 ± 2618.50	63426.22 ± 2618.50
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d70000	19.77 ± 0.09	34.20 ± 17.97
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d100000	972.62 ± 14.11		93052.63 ± 1199.68	93051.26 ± 1199.68	93052.67 ± 1199.68
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d100000	19.13 ± 0.45	21.33 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d130000	930.19 ± 20.29		126303.23 ± 3034.33	126301.86 ± 3034.33	126303.29 ± 3034.32
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d130000	18.34 ± 0.17	13.67 ± 8.96

today:

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d1000	844.91 ± 250.09		2449.36 ± 931.07	2429.47 ± 931.07	2449.40 ± 931.08
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d1000	25.20 ± 0.45	26.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d10000	1320.27 ± 17.46		7588.38 ± 123.31	7568.49 ± 123.31	7588.43 ± 123.31
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d10000	24.78 ± 0.05	25.67 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d15000	1398.10 ± 3.06		10312.15 ± 18.60	10292.26 ± 18.60	10312.20 ± 18.60
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d15000	23.78 ± 0.88	25.62 ± 0.88
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d20000	1110.99 ± 216.48		17613.41 ± 2919.23	17593.53 ± 2919.23	17613.46 ± 2919.24
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d20000	24.22 ± 0.05	25.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d30000	1421.89 ± 3.20		19472.32 ± 100.78	19452.43 ± 100.78	19472.36 ± 100.78
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d30000	23.57 ± 0.03	24.33 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d40000	1391.89 ± 10.45		26410.38 ± 383.42	26390.49 ± 383.42	26410.43 ± 383.42
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d40000	22.83 ± 0.13	24.00 ± 0.00
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d70000	1276.75 ± 7.37		49667.62 ± 404.97	49647.73 ± 404.97	49667.66 ± 404.97
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d70000	21.35 ± 0.05	22.67 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d100000	1186.91 ± 11.80		76158.60 ± 830.15	76138.71 ± 830.15	76158.65 ± 830.15
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d100000	19.85 ± 0.07	21.33 ± 0.47
deepseek-ai/DeepSeek-V4-Flash	pp1000 @ d130000	1098.38 ± 27.00		106745.53 ± 2722.10	106725.64 ± 2722.10	106745.57 ± 2722.10
deepseek-ai/DeepSeek-V4-Flash	tg128 @ d130000	18.47 ± 0.01	13.67 ± 8.96

corbett_korbett · June 6, 2026, 12:15am

I get around that for token gen on this one minimax m2.7 quant that feels a lot more intelligent than the cyanwiki awq 4bit quant. Most intelligent quant of this model I have used it was made for a dual strix halo setup but works great on sparks since its just a more intelligently made quant ayysasha/MiniMax-M2.7-AWQ-G32-STRIX-2H · Hugging Face

mahesh.prem.tysons.ai · June 6, 2026, 7:42pm

tried pointing a gemma model to understand this ..even that is very confused .. can someeone pls help summarize what I should do .. just a few concrete pointers would help

am i to git clone this repo ? feat_custom_repos

or which one is latest add-deepseek-v4-flash-recipe

Whats the final build args pls

./build-and-copy.sh --vllm-repo `https://github.com/jasl/vllm.git\` --vllm-ref dda4668b59567416f86956cfe7bbc1eab371a61e --rebuild-vllm -t vllm-node-dsv4 -c or

./build-and-copy.sh --vllm-repo `https://github.com/jasl/vllm.git\` --vllm-ref

dda4668b59567416f86956cfe7bbc1eab371a61e

--rebuild-vllm

and which recipe pls

bernisse · June 6, 2026, 9:10pm

Here is what I did, there are easier ways but this was straight forward for me and has been working fine. I had help from many on this forum which I have already recognized in earlier posts. This setup works great for me and this is the best model I have used on the Spark to date.

Create a Dockerfile in an empty folder with these two lines in it:

FROM aidendle94/sparkrun-vllm-ds4-gb10:production-ready
WORKDIR /workspace

Build the image using a docker command from the folder with the Dockerfile, after the -t you can name the image whatever you want:

docker build . -t vllm-node-dsv4

At this point you have a compatible vllm image that will work with the community scripts from @eugr_nv.

Copy the image over to your other node, I used the build-and-copy.sh script, after the -t use whatever name you named your image:

./build-and-copy.sh -t vllm-node-dsv4 —no-build -c

Then you can just use the recipe in this thread from @wolttam utilizing the container name you built previously. Do not forget to use the --no-ray parameter because of memory constraints.

bjk110 · June 7, 2026, 12:02am

github.com/bjk110/spark_vllm_docker

README.md

main

# vLLM Spark — Unified Serving for DGX Spark (GB10)

Unified vLLM serving configuration for NVIDIA DGX Spark (GB10), supporting two
topologies from the same repo / Dockerfile / compose file:

- **Single Spark** (default, zero RDMA setup) — one GB10 box, TP=1.
- **Dual Spark + 200 Gbps RoCE/IB** — two GB10 boxes, Ray, TP=2.

Pick the topology by setting `CLUSTER_MODE=single` (default) or
`CLUSTER_MODE=dual-rdma` in your `.env`. See [`Quick Start`](#quick-start) below.

For release-by-release detail and patch-by-patch status, see
[`CHANGELOG.md`](CHANGELOG.md) and [`PATCH_STATUS.md`](PATCH_STATUS.md).

For a high-level overview of the current repository state and deferred cleanup roadmap,
see [`docs/repository-status.md`](docs/repository-status.md).

## Hardware

| Topology | Node | Role | GPU | Memory | Interconnect |

This file has been truncated. show original

This is the configuration that worked

ttsiodras · June 7, 2026, 12:18pm

Thank you - I finally have DS4 working! I’ve been using single-node antirez’s custom llama so far, since there was no way to run it on both of the SPARKs I work with… But now that I can, there’s no comparison - 11.5 fps against 40+!.. And that’s with the llama benchy’s Project Gutenberg stuff; on coding (where MTP is very good at predicting tokens) it runs even faster, avg-ing 55-60 in my pi.dev sessions - and with very encouraging (intelligence-wise) result. Put simply - this feels very Sonnet-y.

Only issue so far is that I don’t know how to control effort in pi.dev - settings/thinking level shows “Off (no reasoning)” even though I can clearly see the model is thinking its way through.

A quick llama-benchy result:

$ .venv/bin/llama-benchy --base-url http://localhost:8081/v1 --model deepseek-v4-flash --tokenizer /home/coder/.cache/huggingface/hub/models–deepseek-ai–DeepSeek-V4-Flash/snapshots/6976c7ff1b30a1b2cb7805021b8ba4684041f136/ --pp 2048 --tg 2048 --depth 2048
[transformers] PyTorch was not found. Models won’t be available and only tokenizers, configuration and file/data utilities can be used.
llama-benchy (0.3.7)
Date: 2026-06-07 14:13:46
Benchmarking model: deepseek-v4-flash at http://localhost:8081/v1
Concurrency levels: [1]
Loading text from cache: /home/coder/.cache/llama-benchy/cc6a0b5782734ee3b9069aa3b64cc62c.txt
Total tokens available in text corpus: 142813
Warming up…
Warmup (User only) complete. Delta: 4 tokens (Server: 26, Local: 22)
Warmup (System+Empty) complete. Delta: 4 tokens (Server: 26, Local: 22)

Running coherence test…
Coherence test PASSED.
Measuring latency using mode: api…
Average latency (api): 5.04 ms
Running test: pp=2048, tg=2048, depth=2048, concurrency=1
Run 1/3 (batch size 1)…
Run 2/3 (batch size 1)…
Run 3/3 (batch size 1)…
Printing results in MD format:

model test t/s peak t/s ttfr (ms) est_ppt (ms) e2e_ttft (ms)

deepseek-v4-flash pp2048 @ d2048 1955.99 ± 80.21 2102.73 ± 88.23 2097.70 ± 88.23 2102.73 ± 88.23

deepseek-v4-flash tg2048 @ d2048 38.16 ± 3.48 48.67 ± 2.62

llama-benchy (0.3.7)
date: 2026-06-07 14:13:46 | latency mode: api

co-le · June 7, 2026, 5:38pm

Those are awesome numbers! What recipe did you use?

I had a working docker but thought “hey let’s delete it and try vLLM 0.22 which has all of these DS4 specific fixes!” I trusted my luck too much - didn’t work at all (sparks stuck after loading weights).

Enlighten me please =)

ttsiodras · June 7, 2026, 5:48pm

I followed bernisse’s instructions to the letter; meaning…

I used aidendle94’s Docker image (copied to both nodes)
the recipe from wolttam ( higher up in the thread ).
…and launched with --no-ray.

co-le · June 8, 2026, 9:39am

Perfect, I was able to reproduce it as well.

Click to see container build instructions and recipe

DeepSeek V4 Flash — DGX Spark Cluster Setup

Container Build

FROM aidendle94/sparkrun-vllm-ds4-gb10:production-ready
WORKDIR /workspace

On spark1:

docker build . -t vllm-node-dsv4

Recipe

recipe_version: "1"
name: deepseek-v4-flash
description: DeepSeek V4 Flash FP8 on dual DGX Spark TP=2
model: deepseek-ai/DeepSeek-V4-Flash
container: vllm-node-dsv4:latest
cluster_only: true
mods: []
defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 2
  pipeline_parallel: 1
  gpu_memory_utilization: 0.85
  block_size: 256
  max_model_len: 200000
  max_num_batched_tokens: 8192
  max_num_seqs: 2
  served_model_name: deepseek-v4-flash
env:
  HF_HOME: /root/.cache/huggingface
  HF_HUB_OFFLINE: "1"
  TRANSFORMERS_OFFLINE: "1"
  VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
  VLLM_USE_B12X_MOE: "1"
  VLLM_SPARSE_INDEXER_MAX_LOGITS_MB: "256"
  VLLM_TRITON_MLA_SPARSE: "1"
  TORCH_CUDA_ARCH_LIST: 12.1a
  TRITON_CACHE_DIR: "/root/.triton"
  VLLM_ENGINE_READY_TIMEOUT_S: "7200"
command: |
  vllm serve deepseek-ai/DeepSeek-V4-Flash \
      --served-model-name {served_model_name} \
      --host {host} \
      --port {port} \
      --trust-remote-code \
      --tensor-parallel-size {tensor_parallel} \
      --pipeline-parallel-size {pipeline_parallel} \
      --kv-cache-dtype fp8 \
      --block-size {block_size} \
      --enable-prefix-caching \
      --max-model-len {max_model_len} \
      --max-num-seqs {max_num_seqs} \
      --enable-chunked-prefill \
      --max-num-batched-tokens {max_num_batched_tokens} \
      --gpu-memory-utilization {gpu_memory_utilization} \
      --distributed-executor-backend mp \
      --speculative-config '{{"method":"mtp","num_speculative_tokens":2}}' \
      --tokenizer-mode deepseek_v4 \
      --tool-call-parser deepseek_v4 \
      --enable-auto-tool-choice \
      --reasoning-parser deepseek_v4 \
      --reasoning-config '{{"reasoning_parser":"deepseek_v4","reasoning_start_str":"<think>","reasoning_end_str":"</think>"}}' \
      --default-chat-template-kwargs '{{"thinking":true,"preserve_thinking":true}}' \
      --load-format safetensors \
      --nnodes 2

I tried using instanttensor, but it introduced many issues (not in itself, but adding it to the container downgraded NCCL for some reason, leading to stuck nodes).

I’ve been loving DS4 Flash through DeepSeek API and now it’s running GREAT on my 2xSpark.

Running on 2x Spark with above recipe:

Speed from DeepSeek API:

Thanks!

11_p · June 8, 2026, 1:31pm

⚠️ PSA: Prefix caching bug in vLLM V1 engine (PR #44237)
If you’re running with --enable-prefix-caching on the V1 engine (default in recent vLLM), there’s a bug in vllm/v1/core/block_pool.py that causes:
kv_cache_usage_perc only goes up, never down
Linear RSS growth over time
Prefix cache progressively loses effectiveness → more cold starts
Root cause: In _maybe_evict_cached_block(), block.reset_hash() is only called when pop() succeeds. When a hash collision occurs, pop() returns None, reset_hash() is skipped, and the block keeps its stale hash. This causes cache_full_blocks() assertion to fail, preventing new tokens from being cached.
Fix: PR #44237 — move block.reset_hash() outside the conditional so it always executes. Single-file patch, only affects vllm/v1/core/block_pool.py.
Link: [Bugfix] Fix linear host RSS growth under sustained classification load with prefix caching (V1) by Oxygen56 · Pull Request #44237 · vllm-project/vllm · GitHub
Workaround: Manually patch the file in your Docker container (or mount with -v). No other dependencies.

kan11 · June 8, 2026, 4:22pm

I can confirm the symptom. Cache keeps accumulating until 99%, and vLLM struggles until it’s cleared. After patching the image with PR #44237, the cache clears immediately after request completion. DSv4F is running much better at higher context now.

Thank you @aidendle94 for making DSv4F run great on Spark. It is now my main model, even without vision.

a.grmbl · June 8, 2026, 5:38pm

Thanks for the PSA!

Confirm the sympoms - KV cache ratcheting up, near-zero prompt cache hits. Did not crash, but after an hour or so performance severely degraded.

Patch definitely helped, now back to normal.

bernisse · June 8, 2026, 7:24pm

Which VLLM image are you using? I have not seen this with the one I am using from @aidendle94. I am wondering if I should recompile it with this PR.

paxren2020 · June 8, 2026, 7:29pm

Could I please ask you to upload the patched file so I can replace it inside the image?

kan11 · June 9, 2026, 3:25am

Here you go: kanthai/spark-vllm-dsv4f-aidendle94:production-ready-pr44237

It’s on docker hub now. The patched file is already in the image.

Topic		Replies	Views
Deepseek v4 Flash on 2 Nodes DGX Spark / GB10 Projects deepseek	71	7013	June 15, 2026
DeepSeek v4 Flash (Aiden Recipe from Reddit) - 1M token session operational, Cuda 12.1 tailored for DGX Spark GB10 DGX Spark / GB10 deepseek	386	19821	July 16, 2026
DeepSeek-V4-Flash on 4× DGX Spark via vLLM (jasl fork, TP=4, RDMA, MTP) — 49–54 tok/s single-stream, full recipe + the traps DGX Spark / GB10 Projects deepseek	3	736	June 19, 2026
Official NVidia optimized DeepSeek-V4-Flash models? DGX Spark / GB10 deepseek	28	1999	July 11, 2026
Deepseek V4 released DGX Spark / GB10 deepseek	143	17492	May 18, 2026
DeepSeek V4 Flash (1,048,576 Context) on 2x DGX Spark – Custom Sparkrun Recipe DGX Spark / GB10 jetson , deepseek	11	1132	June 14, 2026
Fully custom CUDA-native Deepseek 4 Flash optimized for 1x Spark! antirez/ds4 DGX Spark / GB10 Projects gaming , llama , deepseek	79	9084	July 15, 2026
DeepSeek-V4-Flash-DSpark on 2× DGX Spark (GB10) — big single-stream speed boost (~60-67 tok/s) + 1M context, now with concurrency DGX Spark / GB10 deepseek	101	7025	July 19, 2026
DeepSeekV4-Flash hybrid quant, 1x DGX Spark: antirez's optimized 128 GB MLX recipe ported to vLLM for GB10 DGX Spark / GB10 Projects deepseek	18	2176	May 11, 2026
Best model for single Spark? DGX Spark / GB10 spark , deepseek	53	4837	July 14, 2026

model	test	t/s	peak t/s	ttfr (ms)	est_ppt (ms)	e2e_ttft (ms)
deepseek-v4-flash	pp2048 @ d2048	1955.99 ± 80.21		2102.73 ± 88.23	2097.70 ± 88.23	2102.73 ± 88.23
deepseek-v4-flash	tg2048 @ d2048	38.16 ± 3.48	48.67 ± 2.62

DeepSeek-V4-Flash (official FP8) running across 2x DGX Spark — TP=2, MTP, 200K ctx, recipe + numbers

DeepSeek V4 Flash — DGX Spark Cluster Setup

Container Build

Recipe

Related topics